Wireless Vehicle Control of Vehicle Sensor Calibration Environment

ABSTRACT

Sensors coupled to a vehicle are calibrated using a calibration environment that includes sensor targets, and optionally includes a motorized turntable that rotates the vehicle to different orientations, with the sensors capturing data at each orientation. The vehicle&#39;s computer identifies representations of the sensor targets within the data captured by the sensors, determines whether characteristics such as lighting or positioning of the representations of the sensor targets are optimal for calibration, sends signals to lighting systems or motorized target movement systems if necessary to adjust those characteristics, and calibrates the sensor based on the representations of the sensor targets.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.16/457,859, filed on Jun. 28, 2019, entitled, WIRELESS VEHICLE CONTROLOF VEHICLE SENSOR CALIBRATION ENVIRONMENT, which is hereby expresslyincorporated by reference in its entirety.

TECHNICAL FIELD

The present invention generally pertains to calibration of sensors thatare used by vehicles. More specifically, the present invention pertainsto use of a dynamic scene to perform intrinsic and extrinsiccalibrations of various sensors, such as cameras and range sensors, thatare coupled to a vehicle and used by the vehicle to identify itssurroundings.

BACKGROUND

An autonomous vehicle is a motorized vehicle that can navigate without ahuman driver. An exemplary autonomous vehicle includes a plurality ofsensor systems, such as, but not limited to, a camera sensor system, alight detection and ranging (LIDAR) sensor system, or a radio detectionand ranging (RADAR) sensor system, amongst others, wherein theautonomous vehicle operates based upon sensor signals output by thesensor systems. Specifically, the sensor signals are provided to aninternal computing system in communication with the plurality of sensorsystems, wherein a processor executes instructions based upon the sensorsignals to control a mechanical system of the autonomous vehicle, suchas a vehicle propulsion system, a braking system, or a steering system.Such sensors may also be mounted on other vehicles, such as vehiclesthat are used to generate or update street maps as they drive.

A wide range of manufacturing defects or discrepancies can exist invehicles, sensors, and mounting hardware that affixes the sensors to thevehicles. Because of these discrepancies, different sensors mounted todifferent vehicles may capture slightly different data, even when thosevehicles are at the exact same position, and even when the vehicles arebrand new. For example, a lens of one camera may be warped slightly (orinclude some other imperfection) compared to a lens of another camera,one vehicle may include a newer hardware revision or version of aparticular sensor than another, one vehicle's roof may be a fewmillimeters higher or lower than another vehicle's roof, or a skewedscrew used in a mounting structure for a sensor on one vehicle may tiltthe mounting structure slightly. Such imperfections and variations inmanufacturing can impact sensor readings and mean that there no twovehicles capture sensor readings in quite the same way, and thus no twovehicles interpret their surroundings via sensor readings in quite thesame way. With use, vehicles can drift even further apart in theirsensor readings due to exposure to the elements, for example throughexposure to heat, rain, dust, frost, rocks, pollution, vehicularcollisions, all of which can further damage or otherwise impact avehicle or its sensor.

Sensors typically capture data and provide results in a standardizedmanner that does not, by itself, test or account for intrinsicproperties of each sensor, such as the position and angle of the sensoror properties of a lens, or for extrinsic relationships between sensorsthat capture data from similar areas. Because of this, it can be unclearwhether a discrepancy in measurements between two vehicles can beattributed to an actual difference in environment or simply differentproperties of vehicle sensors. Because autonomous vehicles are trustedwith human lives, it is imperative that autonomous vehicles have asrobust an understanding of their environments as possible, otherwise avehicle might perform an action that it should not perform, or fail toperform an action that it should perform, either of which can result ina vehicular accident and put human lives at risk. Other sensor-ladenvehicles, such as those that collect data for maps or street-levelimagery, can produce unreliable maps or images if they cannot accountfor the properties of their sensors, which can then in turn confuse bothhuman vehicles and autonomous vehicles that rely on those maps, againrisking human life.

BRIEF DESCRIPTION OF THE DRAWINGS

The above-recited and other advantages and features of the presenttechnology will become apparent by reference to specific implementationsillustrated in the appended drawings. A person of ordinary skill in theart will understand that these drawings only show some examples of thepresent technology and would not limit the scope of the presenttechnology to these examples. Furthermore, the skilled artisan willappreciate the principles of the present technology as described andexplained with additional specificity and detail through the use of theaccompanying drawings in which:

FIG. 1 illustrates an autonomous vehicle and remote computing systemarchitecture.

FIG. 2A illustrates a camera calibration target with a checkerboardpattern on a planar substrate.

FIG. 2B illustrates a camera calibration target with a ArUco pattern ona planar substrate.

FIG. 2C illustrates a camera calibration target with a crosshair patternon a planar substrate.

FIG. 2D illustrates a range sensor calibration target with a trihedralshape.

FIG. 2E illustrates a combined range sensor and camera calibrationtarget with apertures from a planar substrate that are surrounded byvisually recognizable markings.

FIG. 3 illustrates a top-down view of a hallway calibration environmentin which a vehicle traverses a drive path along which the vehicle isflanked by vehicle sensor calibration targets.

FIG. 4 illustrates a perspective view of a dynamic scene calibrationenvironment in which a turntable that is at least partially surroundedby vehicle camera calibration targets rotates a vehicle so that thevehicle can perform intrinsic calibration of its camera sensors.

FIG. 5 illustrates a perspective view of a dynamic scene calibrationenvironment in which a turntable that is at least partially surroundedby combined vehicle range sensor and vehicle camera calibration targetsrotates a vehicle so that the vehicle can perform extrinsic calibrationof its range sensors and camera sensors.

FIG. 6 illustrates a top-down view of a dynamic scene calibrationenvironment in which a turntable that is at least partially surroundedby both vehicle camera calibration targets and combined vehicle rangesensor and vehicle camera calibration targets rotates a vehicle so thatthe vehicle can perform both intrinsic calibration of its camera sensorsand extrinsic calibration of its range sensors and camera sensors.

FIG. 7 illustrates a system architecture of an dynamic scene calibrationenvironment.

FIG. 8A illustrates a top-down view of a turntable with a vehicle guiderail, and a vehicle driving onto the turntable centered relative to thevehicle guide rail.

FIG. 8B illustrates a vehicle having successfully driven onto theturntable centered along the vehicle guide rail, the vehicle thuscentered with respect to the turntable.

FIG. 8C illustrates a top-down view of a turntable with a vehicle guiderail, and a vehicle driving onto the turntable off-center relative tothe vehicle guide rail.

FIG. 8D illustrates a vehicle having driven partially onto the turntablewhile off-center relative to the vehicle guide rail, the vehicle guiderail guiding the vehicle's path to center the vehicle with respect tothe turntable.

FIG. 9 is a flow diagram illustrating operation of a calibrationenvironment.

FIG. 10 is a flow diagram illustrating operations for intrinsiccalibration of a vehicle sensor using a dynamic scene.

FIG. 11 is a flow diagram illustrating operations for extrinsiccalibration of two sensors in relation to each other using a dynamicscene.

FIG. 12 is a flow diagram illustrating operations for interactionsbetween the vehicle and the turntable.

FIG. 13 is a flow diagram illustrating operations for detection of, andcompensation for, a sloped turntable surface.

FIG. 14A is a flow diagram illustrating operations for interactionsbetween the vehicle and a lighting system.

FIG. 14B is a flow diagram illustrating operations for interactionsbetween the vehicle and a target control system.

FIG. 15 shows an example of a system for implementing certain aspects ofthe present technology.

DETAILED DESCRIPTION

Various examples of the present technology are discussed in detailbelow. While specific implementations are discussed, it should beunderstood that this is done for illustration purposes only. A personskilled in the relevant art will recognize that other components andconfigurations may be used without parting from the spirit and scope ofthe present technology. In some instances, well-known structures anddevices are shown in block diagram form in order to facilitatedescribing one or more aspects. Further, it is to be understood thatfunctionality that is described as being carried out by certain systemcomponents may be performed by more or fewer components than shown.

The disclosed technologies address a need in the art for improvements tovehicle sensor calibration technologies. Use of a dynamic calibrationscene with an automated turntable or carousel system improves thefunctioning of sensor calibration by improving runtime-efficiency,space-efficiency, comprehensiveness of calibration, and consistency ofvehicle sensor calibration. The described vehicle sensor calibrationtechnologies ultimately transform vehicle sensors from an uncalibratedstate to a calibrated state. The described vehicle sensor calibrationtechnologies are implemented using a vehicle, the vehicle's sensors, aturntable and other potential components of a dynamic scene, one or morecomputing devices associated with the other components, each of which isintegral at least one embodiment of the vehicle sensor calibrationtechnologies.

Sensors coupled to a vehicle are calibrated using a calibrationenvironment that includes sensor targets, and optionally includes amotorized turntable that rotates the vehicle to different orientations,with the sensors capturing data at each orientation. The vehicle'scomputer identifies representations of the sensor targets within thedata captured by the sensors, determines whether characteristics such aslighting or positioning of the representations of the sensor targets areoptimal for calibration, sends signals to lighting systems or motorizedtarget movement systems if necessary to adjust those characteristics,and calibrates the sensor based on the representations of the sensortargets.

FIG. 1 illustrates an autonomous vehicle and remote computing systemarchitecture.

The autonomous vehicle 102 can navigate about roadways without a humandriver based upon sensor signals output by sensor systems 180 of theautonomous vehicle 102. The autonomous vehicle 102 includes a pluralityof sensor systems 180 (a first sensor system 104 through an Nth sensorsystem 106). The sensor systems 180 are of different types and arearranged about the autonomous vehicle 102. For example, the first sensorsystem 104 may be a camera sensor system and the Nth sensor system 106may be a Light Detection and Ranging (LIDAR) sensor system. Otherexemplary sensor systems include radio detection and ranging (RADAR)sensor systems, Electromagnetic Detection and Ranging (EmDAR) sensorsystems, Sound Navigation and Ranging (SONAR) sensor systems, SoundDetection and Ranging (SODAR) sensor systems, Global NavigationSatellite System (GNSS) receiver systems such as Global PositioningSystem (GPS) receiver systems, accelerometers, gyroscopes, inertialmeasurement units (IMU), infrared sensor systems, laser rangefindersystems, ultrasonic sensor systems, infrasonic sensor systems,microphones, or a combination thereof. While four sensors 180 areillustrated coupled to the autonomous vehicle 102, it should beunderstood that more or fewer sensors may be coupled to the autonomousvehicle 102.

The autonomous vehicle 102 further includes several mechanical systemsthat are used to effectuate appropriate motion of the autonomous vehicle102. For instance, the mechanical systems can include but are notlimited to, a vehicle propulsion system 130, a braking system 132, and asteering system 134. The vehicle propulsion system 130 may include anelectric motor, an internal combustion engine, or both. The brakingsystem 132 can include an engine brake, brake pads, actuators, and/orany other suitable componentry that is configured to assist indecelerating the autonomous vehicle 102. In some cases, the brakingsystem 132 may charge a battery of the vehicle through regenerativebraking. The steering system 134 includes suitable componentry that isconfigured to control the direction of movement of the autonomousvehicle 102 during navigation.

The autonomous vehicle 102 further includes a safety system 136 that caninclude various lights and signal indicators, parking brake, airbags,etc. The autonomous vehicle 102 further includes a cabin system 138 thatcan include cabin temperature control systems, in-cabin entertainmentsystems, etc.

The autonomous vehicle 102 additionally comprises an internal computingsystem 110 that is in communication with the sensor systems 180 and thesystems 130, 132, 134, 136, and 138. The internal computing systemincludes at least one processor and at least one memory havingcomputer-executable instructions that are executed by the processor. Thecomputer-executable instructions can make up one or more servicesresponsible for controlling the autonomous vehicle 102, communicatingwith remote computing system 150, receiving inputs from passengers orhuman co-pilots, logging metrics regarding data collected by sensorsystems 180 and human co-pilots, etc.

The internal computing system 110 can include a control service 112 thatis configured to control operation of the vehicle propulsion system 130,the braking system 208, the steering system 134, the safety system 136,and the cabin system 138. The control service 112 receives sensorsignals from the sensor systems 180 as well communicates with otherservices of the internal computing system 110 to effectuate operation ofthe autonomous vehicle 102. In some embodiments, control service 112 maycarry out operations in concert one or more other systems of autonomousvehicle 102.

The internal computing system 110 can also include a constraint service114 to facilitate safe propulsion of the autonomous vehicle 102. Theconstraint service 116 includes instructions for activating a constraintbased on a rule-based restriction upon operation of the autonomousvehicle 102. For example, the constraint may be a restriction uponnavigation that is activated in accordance with protocols configured toavoid occupying the same space as other objects, abide by traffic laws,circumvent avoidance areas, etc. In some embodiments, the constraintservice can be part of the control service 112.

The internal computing system 110 can also include a communicationservice 116. The communication service can include both software andhardware elements for transmitting and receiving signals from/to theremote computing system 150. The communication service 116 is configuredto transmit information wirelessly over a network, for example, throughan antenna array that provides personal cellular (long-term evolution(LTE), 3G, 4G, 5G, etc.) communication.

In some embodiments, one or more services of the internal computingsystem 110 are configured to send and receive communications to remotecomputing system 150 for such reasons as reporting data for training andevaluating machine learning algorithms, requesting assistance fromremoting computing system or a human operator via remote computingsystem 150, software service updates, ridesharing pickup and drop offinstructions etc.

The internal computing system 110 can also include a latency service118. The latency service 118 can utilize timestamps on communications toand from the remote computing system 150 to determine if a communicationhas been received from the remote computing system 150 in time to beuseful. For example, when a service of the internal computing system 110requests feedback from remote computing system 150 on a time-sensitiveprocess, the latency service 118 can determine if a response was timelyreceived from remote computing system 150 as information can quicklybecome too stale to be actionable. When the latency service 118determines that a response has not been received within a threshold, thelatency service 118 can enable other systems of autonomous vehicle 102or a passenger to make necessary decisions or to provide the neededfeedback.

The internal computing system 110 can also include a user interfaceservice 120 that can communicate with cabin system 138 in order toprovide information or receive information to a human co-pilot or humanpassenger. In some embodiments, a human co-pilot or human passenger maybe required to evaluate and override a constraint from constraintservice 114, or the human co-pilot or human passenger may wish toprovide an instruction to the autonomous vehicle 102 regardingdestinations, requested routes, or other requested operations.

The internal computing system 110 can, in some cases, include at leastone computing system 1500 as illustrated in or discussed with respect toFIG. 15, or may include at least a subset of the components illustratedin FIG. 15 or discussed with respect to computing system 1500.

As described above, the remote computing system 150 is configured tosend/receive a signal from the autonomous vehicle 140 regardingreporting data for training and evaluating machine learning algorithms,requesting assistance from remote computing system 150 or a humanoperator via the remote computing system 150, software service updates,rideshare pickup and drop off instructions, etc.

The remote computing system 150 includes an analysis service 152 that isconfigured to receive data from autonomous vehicle 102 and analyze thedata to train or evaluate machine learning algorithms for operating theautonomous vehicle 102. The analysis service 152 can also performanalysis pertaining to data associated with one or more errors orconstraints reported by autonomous vehicle 102.

The remote computing system 150 can also include a user interfaceservice 154 configured to present metrics, video, pictures, soundsreported from the autonomous vehicle 102 to an operator of remotecomputing system 150. User interface service 154 can further receiveinput instructions from an operator that can be sent to the autonomousvehicle 102.

The remote computing system 150 can also include an instruction service156 for sending instructions regarding the operation of the autonomousvehicle 102. For example, in response to an output of the analysisservice 152 or user interface service 154, instructions service 156 canprepare instructions to one or more services of the autonomous vehicle102 or a co-pilot or passenger of the autonomous vehicle 102.

The remote computing system 150 can also include a rideshare service 158configured to interact with ridesharing applications 170 operating on(potential) passenger computing devices. The rideshare service 158 canreceive requests to be picked up or dropped off from passengerridesharing app 170 and can dispatch autonomous vehicle 102 for thetrip. The rideshare service 158 can also act as an intermediary betweenthe ridesharing app 170 and the autonomous vehicle wherein a passengermight provide instructions to the autonomous vehicle to 102 go around anobstacle, change routes, honk the horn, etc.

The rideshare service 158 as depicted in FIG. 1 illustrates a vehicle102 as a triangle en route from a start point of a trip to an end pointof a trip, both of which are illustrated as circular endpoints of athick line representing a route traveled by the vehicle. The route maybe the path of the vehicle from picking up the passenger to dropping offthe passenger (or another passenger in the vehicle), or it may be thepath of the vehicle from its current location to picking up anotherpassenger.

The remote computing system 150 can, in some cases, include at least onecomputing system 1500 as illustrated in or discussed with respect toFIG. 15, or may include at least a subset of the components illustratedin FIG. 15 or discussed with respect to computing system 1500.

FIG. 2A illustrates a camera calibration target with a checkerboardpattern on a planar substrate.

The sensor calibration target 220A illustrated in FIG. 2A is a planarboard made from a substrate 205, with a pattern 210A printed, stamped,engraved, imprinted, or otherwise marked thereon. The pattern 210A ofFIG. 2A is a checkerboard pattern. The substrate 205 may be paper,cardboard, plastic, metal, foam, or some combination thereof. Thesubstrate 205 may in some cases include a translucent or transparentsurface upon which the pattern 210A is printed, and which a light sourcemay provide illumination through. The substrate 205 may in some casesinclude a retroreflective surface upon which the pattern 210A isprinted. The retroreflective property of the surface may be inherent tothe material of the substrate 205 or may be a separate layer applied tothe surface of the substrate, for example by adhering a retroreflectivematerial to the substrate 205 or by painting (e.g., via a brush, roller,or aerosol spray) the substrate 205 with a retroreflective paint. Areflective or retroreflective property may in some cases improvedetection using radar, lidar, or other EmDAR sensors. The material andshape of the substrate 205 may also be selected such that the materialand/or shape produces a high amount of acoustic resonance or acousticresponse to improve detection using SONAR or SODAR sensors. In somecases, the substrate 205, and therefore the target 200A, may be concave,convex, otherwise curved, or some combination thereof. The substrate 205may in some cases include devices, such as speakers, heat sources, orlight sources, that allow improved detection by microphones, infraredsensors, or cameras, respectively.

The sensor calibration target 220A illustrated in FIG. 2A is useful forcalibration of a camera of the vehicle, or other sensor that capturesvisual data. In particular, a camera with a pattern/image/featurerecognition system running on computer system 110 can identify thecheckerboard pattern 210A of FIG. 2A, and can identify pointsrepresenting vertices between the dark (black) and light (white)checkers. By drawing lines connecting these points, the camera andcomputer system 110 can generate a grid. If the camera has a wide-anglelens, such as a fisheye lens or a barrel lens, the resulting grid willbe warped so that some checkers will appear curved rather than straight,and so that checkers near the edges of the camera's point of view willappear more squashed, while checkers near the center of the camera'spoint of view will appear larger and more even. A rectilinear lensprovides a similar, is opposite, effect. Based on prior knowledge ofwhat the checkerboard pattern and resulting grid should look like, andits original dimensions, compared against what its representation lookslike as captured by the camera, the camera and computing system 110 mayidentify the effect of the lens and counteract it. The camera andcomputing system 110 may also identify other parameters of the camerathis way, such as position parameters (x, y, z, roll, pitch, yaw), anylens color to be filtered out, any crack or defect in the lens to befiltered out, or a combination thereof.

The sensor calibration target 220A illustrated in FIG. 2A is useful fordetection by, and calibration of, a range sensor of the vehicle, such asa LIDAR, SONAR, SODAR, or radar sensor of the vehicle, at least in thatthe shape of the planar substrate 205 can be detected by the rangesensor. For example, flat planar vision targets such as the target 220Acan be detected by lidar by relying on planar geometry estimates andusing the returned intensity. While FIG. 2A illustrates a square orrectangular substrate 205, the substrate 205 may be circular,semicircular, ellipsoidal, triangular, quadrilateral (trapezoid,parallelogram), pentagonal, hexagonal, heptagonal, octagonal, nonagonal,decagonal, otherwise polygonal, or some combination thereof.

FIG. 2B illustrates a camera calibration target with a ArUco pattern ona planar substrate.

The sensor calibration target 220B illustrated in FIG. 2B, like thesensor calibration target 220A illustrated in FIG. 2A, includes a planarboard made from a substrate 205, with a pattern 210B printed, stamped,engraved, imprinted, or otherwise marked thereon. The pattern 210Billustrated in FIG. 2B is an ArUco marker pattern, which includes blackborder and an inner binary matrix/grid (e.g., each square is dark/blackor light/white) which determines its identifier.

By detecting the AuUco pattern, the camera and computing system 110 ofthe vehicle also identify a grid, similarly to the checkerboard, thoughpotentially with fewer points, as some areas of the ArUco pattern mayinclude contiguous dark/black squares or contiguous light/white squares.By identifying the grid from the representation of the ArUco targetcaptured by the camera (e.g. with lens distortion such as parabolicdistortion), and comparing it to a known reference image of the ArUcopattern (e.g., without any distortion), any distortions or otherdifferences may be identified, and appropriate corrections may begenerated to counteract these distortions or other differences.

The substrate 205 of FIG. 2B may include or be coated with anypreviously-discussed substrate material and may be warped or shaped inany manner or include any devices discussed with respect to thesubstrate 205 of FIG. 2A, and therefore may be detected by, and beuseful to calibrate a range sensor of the vehicle, such as a LIDAR,SONAR, SODAR, or radar sensor of the vehicle, and may be detected by amicrophone or infrared sensor of the vehicle as well.

FIG. 2C illustrates a camera calibration target with a crosshair patternon a planar substrate.

The sensor calibration target 220C illustrated in FIG. 2B, like thesensor calibration target 220A illustrated in FIG. 2A, includes a planarboard made from a substrate 205, with a pattern 210C printed, stamped,engraved, imprinted, or otherwise marked thereon. The pattern 210Cillustrated in FIG. 2C is an crosshair marker pattern, which includesfour dark/black lines and two dark/black circles centered on alight/white background, and with a gap in the dark/black lines near butnot at the center, effectively leaving a “+” symbol in the very center.

The camera and computing system 110 can identify the target 200C byidentifying the circles, the lines, and the intersections of the same.In doing so, the crosshair pattern is identified from the representationof the target 220C captured by the camera (e.g. with lens distortion),and can be compared it to a known reference image of the crosshairpattern target 200C (e.g., without any distortion). As with thecheckerboard and ArUco targets, any distortions or other differences maybe identified, and appropriate corrections may be generated tocounteract these distortions or other differences.

The substrate 205 of FIG. 2C may include or be coated with anypreviously-discussed substrate material and may be warped or shaped inany manner or include any devices discussed with respect to thesubstrate 205 of FIG. 2A, and therefore may be detected by, and beuseful to calibrate a range sensor of the vehicle, such as a LIDAR,SONAR, SODAR, or radar sensor of the vehicle, and may be detected by amicrophone or infrared sensor of the vehicle as well.

While the only patterns 210A-C discussed with respect to camera sensortargets are checkerboard patterns 210A, ArUco patterns 210B, andcrosshair patterns 210C, other patterns that are not depicted canadditionally or alternatively be used. For example, bar codes or quickresponse (QR) codes may be used as patterns 210 that can be recognizedusing the camera and computing device 110.

FIG. 2D illustrates a range sensor calibration target with a trihedralshape.

The sensor calibration target 220 of FIG. 2D is made to be detected by,and use for calibration of, a range sensor, such as a radar sensor (orLIDAR, SONAR, or SODAR) of the vehicle. In particular, the sensorcalibration target 220 of FIG. 2D is trihedral in shape, and may be aconcave or convex trihedral corner, essentially a triangular corner of acube. Alternately, it may be a different shape, such as a corner of adifferent polyhedron (at least portions of all faces of the polyhedronthat touch a particular vertex). Such a shape, especially when concaveand where perpendicular faces are included, produces a particularlystrong radar echo and thus a particularly strong radar cross section(RCS) because incoming radio waves are backscattered by multiplereflection. The RCS of the trihedral corner target is given by:σ=(4·π·a⁴)/(3·λ²), where a is the length of the side edges of the threetriangles, and λ is a wavelength of radar transmitter.

The substrate 205 of the sensor calibration target 220 of FIG. 2D mayinclude or be coated with any previously-discussed substrate materialand may be warped or shaped in any manner, or include any devices,discussed with respect to the substrate 205 of FIG. 2A. In oneembodiment, the substrate 205 of the sensor calibration target 220 ofFIG. 2D may be metal, may be electrically conductive, may be reflectiveor retroreflective, or some combination thereof.

FIG. 2E illustrates a combined range sensor and camera calibrationtarget with apertures from a planar substrate that are surrounded byvisually recognizable markings.

The combined range sensor and camera calibration target 250 of FIG. 2Eincludes multiple apertures 225 in a substrate 205, and includes visualmarkings or patterns 230 at least partially surrounding each aperture.In particular, the target 250 includes four symmetrical circular orellipsoid apertures 225 from a light/white substrate 205 withsymmetrical dark/black circular or ellipsoid rings 230 around theapertures, with three of the apertures/ring combinations being a firstsize (e.g., apertures being 30 cm in diameter and the correspondingrings slightly larger) and a fourth aperture/ring combination 240 beinga second size (e.g., the aperture being 26 cm in diameter and thecorresponding ring slightly larger). The rings around the three largerapertures 225 are likewise larger than the ring around the smalleraperture 240. In some cases, one may be larger than the other three, ortwo may be larger or smaller than the other two, or some combinationthereof. The apertures 225/240 may alternately be referred to ascutouts, holes, voids, orifices, vents, openings, gaps, perforations,interstices, discontinuities or some combination thereof. In some cases,different types of surface discontinuities may be used instead of or inaddition to the apertures 225/240, such as raised surfaces or bumps thatcan also be detected by depth/range sensors such as radar or lidar.

The combined range sensor and camera calibration target 250 of FIG. 2Ealso includes additional markings or patterns at certain edges of thesubstrate, identified as target identification (ID) markers 235. Theparticular combined range sensor and camera calibration target 250 ofFIG. 2E includes target identification (ID) markers 235 on the two sidesof the substrate opposite the smaller aperture/ring combination 240, butother combined range sensor and camera calibration targets 250 may haveone, two, three, or four target identification (ID) markers 235 alongany of the four sides of the square substrate, or may have targetidentification (ID) markers 235 in an amount up to the number of sidesof the polygonal substrate 205 where the substrate 205 is shaped like anon-quadrilateral polygon. That is, if the substrate 205 is an octagon,each combined combined range sensor and camera calibration target 250may have anywhere from zero to eight target identification (ID) markers235. Different patterns of target identification (ID) markers 235 arefurther visible in FIG. 5.

The substrate 205 of FIG. 2E may include or be coated with anypreviously-discussed substrate material and may be warped or shaped inany manner or include any devices discussed with respect to thesubstrate 205 of FIG. 2A, and therefore may be detected by, and beuseful to calibrate a range sensor of the vehicle, such as a LIDAR,SONAR, SODAR, or radar sensor of the vehicle, and may be detected by amicrophone or infrared sensor of the vehicle as well.

In some cases, the combined range sensor and camera calibration target250 may have more or fewer apertures and corresponding visual markingsthan the four apertures and corresponding visual markings illustrated inFIG. 2E.

Additional targets not depicted in FIG. 2A-2E may also be possible forcalibration of different types of vehicle sensors. For example, targetsfor intrinsic or extrinsic calibration of infrared cameras or othertypes of infrared sensors of a vehicle 102 may include patterns formedusing heating elements, optionally in front of, behind, or beside visualmarkings or substrate apertures or speakers for extrinsic calibrationwith cameras or range sensors or microphones, respectively. Targets forintrinsic or extrinsic calibration of microphones or other types ofaudio sensors of a vehicle 102 may include speakers or patterns formedusing speakers, optionally in front of, behind, or beside visualmarkings or substrate apertures or heating elements for extrinsiccalibration with cameras or range sensors or infrared sensors,respectively.

FIG. 3 illustrates a top-down view of a hallway calibration environmentin which a vehicle traverses a drive path along which the vehicle isflanked by vehicle sensor calibration targets.

The hallway calibration environment 300, which may also be referred toas a tunnel calibration environment, includes a thoroughfare 305 throughwhich a vehicle 102 drives, the thoroughfare 305 flanked on either sideby targets detectable by the sensors 180 of the vehicle 102. Thethoroughfare 305 may also be referred to as the drive path, the drivechannel, the hallway, or the tunnel. Some of the targets are arranged ina left target channel 310 that is to the left of the vehicle 102 as thevehicle 102 traverses the thoroughfare 305. Others of the targets arearranged in a right target channel 315 that is to the right of thevehicle 102 as the vehicle 102 traverses the thoroughfare 305. In FIG.3, the targets in the left target channel 310 and right target channel315 are all checkerboard camera targets 200A as illustrated in FIG. 2A,but they may include any other type of target discussed herein that isused to calibrate any vehicle sensor or combination of vehicle sensors.The left target channel 310 and right target channel 315 may include acombination of different target types, similarly to the calibrationenvironment of FIG. 6; the targets need not all be of a uniform type asillustrated in FIG. 3.

The vehicle 102 drives along the thoroughfare 305, stopping afterincremental amounts, for example, every foot, every N feet, every meter,or every N meters, where N is a number greater than zero, such as 1, 2,3, 4, 5, 6, 7, 8, 9, or 10. At each stop, the vehicle 102 captures datausing each of its vehicle sensors, or at least each of the vehiclesensors that it intends to calibrate. The vehicle 102 stopping helpsprevent issues caused by sensors running while the vehicle 102 is inmotion, such as motion blur or rolling shutter issues in cameras. Thevehicle 102 stopping also ensures that sensors can capture data whilethe vehicle 102 is in the same position, which may be important forextrinsic calibration of two or more sensors with respect to each otherso that a location within data gathered by a first vehicle sensor (e.g.,a range sensor such as a LIDAR or radar sensor) can be understood tocorrespond to a location within data gathered by a second vehicle sensor(e.g., a camera). The vehicle 102 may in some cases traverse thethoroughfare 305 multiple times, for example N times in each direction,where N is, for example, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14,15, 16, 17, 18, 19, or 20.

The sensor targets illustrated in FIG. 3 are illustrated as each mountedon separate easel-style stands. Other types of stands are also possible,such as the type illustrated in FIG. 4 and FIG. 5. Furthermore, multiplesensor targets, of the same type or of different types may be supportedby each stand (as in FIG. 4 and FIG. 5) even though this is notillustrated in FIG. 3.

The sensor targets illustrated in FIG. 3 are illustrated such that someare positioned closer to the thoroughfare 305 while some are positionedfarther from the thoroughfare 305. Additionally, while some targets inFIG. 3 are facing a direction perpendicular to the thoroughfare 305,others are angled up or down with respect to the direction perpendicularto the thoroughfare 305. While the sensor targets illustrated in FIG. 3all appear to be at the same height and all appear to not be rotatedabout an axis extending out from the surface of the target, it should beunderstood that the sensor targets may be positioned at differentheights and may be rotated about an axis extending out from the surfaceof the target as in the targets of FIG. 4 and FIG. 5. Together, thedistance from the thoroughfare 305, the direction faced relative to thethoroughfare 305, the clustering of targets, the height, and therotation about an axis extending out from the surface of the target mayall be varied and modified to provide better intrinsic and extrinsiccalibration. That is, these variations assist in intrinsic calibrationin that collection of data with representations of targets in variouspositions, rotations, and so forth ensures that targets are recognizedas they should be by any sensor, even in unusual positions androtations, and that any necessary corrections be performed to datacaptured by sensors after calibration. These variations assist inextrinsic calibration in that the different positions and rotations andso forth provide more interesting targets for range sensors, such aslidar, radar, sonar, or sodar, and allow range sensors to aid ininterpretation of optical data collected by a camera of the vehicle 102.

While the thoroughfare 305 of the hallway calibration environment 300 ofFIG. 3 is a straight path, in some cases it may be a curved path, and byextension the left target channel 310 and right target channel 315 maybe curved to follow the path of the thoroughfare 305.

While the hallway calibration environment 300 is effective in providingan environment with which to calibrate the sensors 180 of the vehicle102, it is inefficient in some ways. The hallway calibration environment300 is not space efficient, as it occupies a lot of space. Such ahallway calibration environment 300 is best set up indoors so thatlighting can be better controlled, so the hallway calibrationenvironment 300 requires a large indoor space, and by extension, a lotof light sources, which is not energy-efficient or cost-efficient.Because of how much space the hallway calibration environment 300 takesup, it is more likely to have to be taken down and set back up again,affecting consistency of calibration between different vehicles whosesensors are calibrated at different times. Further, because the setup ofthe hallway calibration environment 300 requires the vehicle 102 todrive through it, different vehicles 102 might be aligned slightlydifferently in the thoroughfare 102, and might drive a slightlydifferent path through the thoroughfare 102, and might stop at slightlydifferent spots and/or frequencies along the drive, due to manufacturingdifferences in the vehicle 102 and due to human error in setting thevehicle 102 up, all of which affects consistency of the calibration.Trying to correct for all of these potential inconsistencies, andturning the vehicle around to move it through the hallway calibrationenvironment 300 multiple times, is time and labor intensive, making thehallway calibration environment 300 time-inefficient. Additionally,because the targets are primarily to the left and right sides of thevehicle 102 hallway calibration environment 300, vehicle sensors mightnot be as well calibrated in the regions to the front and rear of thevehicle. Using a thoroughfare 305 with some turns can help alleviatethis, but again causes the hallway calibration environment 300 to takeup more space, increasing space-inefficiency.

FIG. 4 illustrates a perspective view of a dynamic scene calibrationenvironment in which a turntable that is at least partially surroundedby vehicle camera calibration targets rotates a vehicle so that thevehicle can perform intrinsic calibration of its camera sensors.

The dynamic scene calibration environment 400 of FIG. 4 includes amotorized turntable 405 with a platform 420 that rotates about a base425. In some cases, the platform 420 may be raised above thefloor/ground around the turntable 405, with the base 425 graduallyinclined up to enable the vehicle 120 to drive up the base 425 and ontothe platform 420, or to drive off of the platform 420 via the base 425.A vehicle 102 drives onto the platform 420 of the turntable 405, and themotors actuate to rotate platform 420 of the turntable 405 about thebase 425, and to thereby rotate the vehicle 102 relative to the base 425(and therefore relative to the floor upon which the base 425 rests).While the arrows on the turntable 405 show a counter-clockwise rotationof the platform 420, it should be understood that the platform 420 ofthe motorized turntable 405 can be actuated to rotate clockwise as well.The turntable 405 is at least partially surrounded by targets mounted onstands 410. In FIG. 4, the illustrated targets are allcheckerboard-patterned camera calibration targets 200A as depicted inFIG. 2A, allowing for calibration of cameras of the vehicle 102. Inother embodiments, such as in FIG. 5, the targets around the motorizedturntable may include any other type of target discussed herein that isused to calibrate any vehicle sensor or combination of vehicle sensors.

In one embodiment, the platform 420 of the motorized turntable 405 maybe rotated by predetermined intervals (measured in degrees/radians or anamount at a time), for example intervals of ten degrees, in betweenpoint the turntable stops so that the vehicle 102 can capture data withits sensors 180. The platform 420 of the motorized turntable 405 canstart and stop in this manner, and can eventually perform a full 360degree rotation in this manner. The motorized turntable 405 may in somecases perform multiple full 360 degree rotations in one or both rotationdirections (clockwise and counterclockwise), for example N rotations ineach rotation direction, where N is, for example, 1, 2, 3, 4, 5, 6, 7,8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, or 20.

FIG. 5 illustrates a perspective view of a dynamic scene calibrationenvironment in which a turntable that is at least partially surroundedby combined vehicle range sensor and vehicle camera calibration targetsrotates a vehicle so that the vehicle can perform extrinsic calibrationof its range sensors and camera sensors.

The dynamic scene calibration environment 500 of FIG. 5 includes thesame motorized turntable 405 as in FIG. 4. A vehicle 102 drives onto theplatform 420 of the turntable 405, and the motors actuate to rotate theplatform 420 of the turntable 405, and thereby rotate the platform 420(and the vehicle 102) about the base 425, clockwise, counter-clockwise,or one then the other. The turntable 405 is at least partiallysurrounded by targets mounted on stands 410. In FIG. 5, the illustratedtargets include both a set of combined range/camera extrinsiccalibration targets 250 as depicted in FIG. 2E and a set of trihedralradar calibration targets 220 as depicted in FIG. 2D. Each stand 410mounts two combined range/camera extrinsic calibration targets 250 andone trihedral radar calibration targets 220, which in some cases may bea known distance from the combined range/camera extrinsic calibrationtargets 250 on the stand 410, permitting extrinsic calibration betweenthe radar and the range sensor (lidar, radar, sonar, sodar) and/orcamera calibrated using the combined range/camera extrinsic calibrationtargets 250.

As the vehicle 102 rotates about the base 425 on the platform 420 of themotorized turntable 405, and/or during stops between rotations, thevehicle 102 and its computer 110 can detect the combined range/cameraextrinsic calibration targets 250 using both its range sensors (e.g.,lidar, etc.) and cameras by detecting the apertures 225 with the rangesensors and the markings 230 around the apertures and the targetidentifier markings 235 with the cameras. In doing so, the vehicle 102and its computer 110 can detect a center of the circular aperture 225easily, since range sensors such as lidar typically provide a pointcloud of depth measurements that can help identify where the widest patsof each circle are. The rings 230 detected by the camera will also havethe same centers as the apertures, so the range sensor and camera knowthey are looking at the exact same locations for each of these centerpoints. Thus, the camera and range sensor may be extrinsicallycalibrated so that their positional awareness of the surroundings of thevehicle 102 can be positionally aligned. The extrinsic calibration may,in some cases, output one or more matrices (e.g., one or moretransformation matrices) used for transforming a camera location to arange sensor location or vice versa, via translation, rotation, or othertransformations in 3D space. Calibration affects interpretation of datacaptured by the sensors after calibration is complete. Thetransformation(s) that are generated during this extrinsic calibrationcan include one or more types of transformations, includingtranslations, stretching, squeezing, rotations, shearing, reflections,perspective distortion, distortion, orthogonal projection, perspectiveprojection, curvature mapping, surface mapping, inversions, lineartransformations, affine transformations, The translational androtational transformations may include modifications to position, angle,roll, pitch, yaw, or combinations thereof. In some cases, specificdistortions may be performed or undone, for example by removingdistortion (e.g., parabolic distortion) caused by use of a specific typeof lens in a camera or other sensor, such as a wide-angle lens or afisheye lens or a macro lens.

The transformation(s) generated by the computer 110 of the vehicle 102may be used for extrinsic calibration of a first sensor (e.g., thecamera) with respect to a second sensor (e.g., LIDAR or RADAR or SONARor SODAR or another range sensor), so that the computer 102 can mappositions identified in the data output from each sensor to the realworld environment around the vehicle 102 (e.g., in the field of view ofthe sensors 180 of the vehicle 102) and relative to each other, based onknown relative positions of features identified within the outputs ofeach sensor. Such features may include the visual markings of thecombined target 250 as identified by the camera, the apertures asidentified by the range sensor, and optionally a trihedral target 220affixed near or on the target 250 as in the environment 500 of FIG. 5.For example, if translation of positions in data captured by the firstsensor to positions in the real world around the vehicle are alreadyclear through intrinsic calibration, but translation of positions indata captured by the second sensor to positions in the real world aroundthe vehicle are already clear through intrinsic calibration (or viceversa), then the transformation generated through this extrinsiccalibration can translate positions in data captured by the secondsensor to positions in the real world around the vehicle based on (1)the position in the real world around the vehicle of the data from thefirst sensor, and (2) the relative positioning of the position in thedata from the first sensor and the position in the data from the secondsensor. Thus, a sensor that has not been intrinsically calibrated canstill be calibrated extrinsically relative to another sensor, and canstill benefit from the increase in accuracy granted by the intrinsiccalibration of that other sensor.

The trihedral targets 220 can also have a known distance from thecombined range/camera extrinsic calibration targets 250, and in somecases specifically from the centers of the apertures 225 and rings 230of the targets 250, allowing extrinsic calibration between the rangesensor (e.g., radar) that recognizes the trihedral targets 220 and therange sensor (e.g., lidar) that recognizes the apertures 225 and thecamera that recognizes the rings/markings 230.

In other embodiments, the targets around the motorized turntable 405 mayinclude any other type of target discussed herein that is used tocalibrate any vehicle sensor or combination of vehicle sensors, such asthe target 200A of FIG. 2A, the target 200B of FIG. 2B, the target 200Cof FIG. 2C, the target 220 of FIG. 2D, the target 250 of FIG. 2E,targets with heating elements detectable by infrared sensors of thevehicle 102, targets with speakers detectable by microphones of thevehicle 102, targets with reflective acoustic properties detectable bySONAR/SODAR/ultrasonic/infrasonic sensors of the vehicle 102, or somecombination thereof.

The stands 410 used in FIG. 3-6 may include any material discussed withrespect to the substrate 205, such as paper, cardboard, plastic, metal,foam, or some combination thereof. In some cases, certain stands may bemade of a plastic such polyvinyl chloride (PVC) to avoid detection bycertain types of range sensors, such as radar, which detect metal betterthan plastic.

In one embodiment, the platform 420 of the motorized turntable 405 maybe rotated about the base 425 by predetermined intervals (measured indegrees/radians or an amount at a time), for example intervals of tendegrees, in between point the turntable stops so that the vehicle 102can capture data with its sensors 180. The intervals may be N degrees,where N is, for example, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14,15, 16, 17, 18, 19, or 20. The platform 420 of the motorized turntable405 can start and stop its rotation via activation and deactivation ofits motor(s) 730 in this manner, and can eventually perform a full 360degree rotation in this manner. The platform 420 of the motorizedturntable 405 may in some cases perform multiple full 360 degreerotations about the base 425 in one or both rotation directions(clockwise and counterclockwise), for example N rotations in eachrotation direction, where N is, for example, 1, 2, 3, 4, 5, 6, 7, 8, 9,10, 11, 12, 13, 14, 15, 16, 17, 18, 19, or 20.

FIG. 6 illustrates a top-down view of a dynamic scene calibrationenvironment in which a turntable that is at least partially surroundedby both vehicle camera calibration targets and combined vehicle rangesensor and vehicle camera calibration targets rotates a vehicle so thatthe vehicle can perform both intrinsic calibration of its camera sensorsand extrinsic calibration of its range sensors and camera sensors.

The dynamic scene calibration environment 600 of FIG. 6 is, in someways, of a combination of the dynamic scene calibration environment 400of FIG. 4 and the dynamic scene calibration environment 500 of FIG. 5 inthat three types of targets are positioned around the motorizedturntable 405. These three types of targets include thecheckerboard-patterned camera calibration targets 200A used in thedynamic scene calibration environment 400 of FIG. 4 and the trihedralradar calibration targets 220 and combined extrinsic calibration targets250 used in the dynamic scene calibration environment 500 of FIG. 5. InFIG. 6, all three types of targets stand separately from each other.

Any of the stands used in FIG. 3-6 may be any type of stands, such aseasel-type stands, tripod-type stands, or the rod stands 410 with widebases seen in FIG. 4 and FIG. 5. Any of the stands used in FIG. 3-6 mayinclude paper, cardboard, plastic, metal, foam, or some combinationthereof as previously discussed. The stands in some cases may includemotors and actuators enabling positions and/or angles of rotation oftargets to be manipulated, for example to be more clearly visible givenlighting conditions supplied by light sources 620, or to access aaregion of a point of view of a particular vehicle sensor from which thatvehicle sensor has not captured enough data and therefore around whichthe vehicle sensor is under-calibrated. In some cases, these motors andtheir actuators may be controlled wirelessly by the vehicle 102 and/orscene surveying system 610 as discussed further with respect to FIG. 7and FIG. 14.

The dynamic scene calibration environment 600 of FIG. 6 also includes ascene surveying system 610, which may include one or more cameras, oneor more range sensors (e.g., radar, lidar, sonar, sodar, laserrangefinder). This may take the form of a robotic total station (RTS).While the vehicle 102, its sensors 180, and the targets are enough toperform intrinsic calibration of sensors to correct for distortions, forexample, and to perform extrinsic calibration of sensors to alignlocations within data captured by different sensors, for example, insome cases more information may be needed to understand how far theselocations within data captured by different sensors are from the vehicleitself. The scene surveying system 610 captures visual and/or range dataof at least a subset of the dynamic scene calibration environment 600,including the motorized turntable 410, at least some of the targets, andthe vehicle 102 itself. Key points on the vehicle 120 may be tracked toidentify the current pose (rotation orientation/position) of the vehicle102 (and therefore of the platform 420 about the base 425). Datacaptured by the scene surveying system 610 can also be sent to thevehicle 102 and used to verify the data captured by the sensors and theintrinsic and extrinsic calibrations performed based on this data. Thedynamic scene calibration environment 600 also includes several lightsources 620 set up around the turntable 405. These light sources 620 mayinclude any type of light source, such as incandescent bulbs, halogenbulbs, light emitting diodes (LEDs), and so forth. In some cases, thelight sources 620 may include stage lights or spotlights. Mirrors and/orlenses may be used to manipulate light from the light sources 620 torelatively evenly illuminate the targets and/or the dynamic scenecalibration environment 600 as a whole. Additional light sources may bepositioned over the turntable 405 or behind some or all targets,especially if targets use a transparent or translucent substrate. Thelight sources 620 are used to improve readings from cameras of thevehicle and of the scene surveying system 610, and can in some cases becontrolled wirelessly by the vehicle 102 and/or scene surveying system610 as discussed further with respect to FIG. 7 and FIG. 14.

By including targets for intrinsic sensor calibration as well as targetsfor extrinsic calibration around the turntable 405, the sensors 180 ofthe vehicle 102 can receive a comprehensive calibration in the dynamicscene calibration environment 600. The dynamic scene calibrationenvironments 400, 500, and 600 of FIGS. 4-6 have several advantages overthe static hallway calibration environment 300 of FIG. 3. First, thedynamic scene calibration environment is more space-efficient, as itonly requires space for the vehicle turntable 405, space for sometargets, light sources 620, and scene surveying system 610. There is noneed to clear space for a thoroughfare 305 or other long path. Becausefewer light sources 620 are needed to light a smaller space like thedynamic scene calibration environment 600 than a larger space like thehallway calibration environment 300, the dynamic scene calibrationenvironment 600 is more energy-efficient in terms of lighting. Becausethe vehicle engine can be turned off after the vehicle is on theturntable 405, the dynamic scene calibration environment 600 is moreenergy-efficient in terms of vehicular power usage. Because changingrotation directions is considerably quicker than performing a U-turn ina vehicle to change driving directions, and because a smaller space ismore likely to remain untouched and therefore won't need take-down andsetup after and before each use, the dynamic scene calibrationenvironment 600 is more time-efficient. Because the target and lightingsetup, and the movement of the vehicle 102, can be better controlled,the calibration results are more consistent across vehicles 102, whichfurther allows calibration configurations of different vehicles 102 tobe more readily compared and outliers identified as potentiallysymptomatic of a sensor defect or a miscalibration issue. Use of guiderailings 810 as illustrated in and discussed with respect to FIG. 8A-8Dfurther increases consistency of calibration by increasing consistencyof positioning of the vehicle 102 along the turntable 405. The dynamicscene calibration environment 600 also allows the sensors 180 of thevehicle 102 to capture data all around the vehicle 102—including in thefront and rear of the vehicle 102, which receive less data in thehallway calibration environment 300.

FIG. 7 illustrates a system architecture of an dynamic scene calibrationenvironment.

The system architecture 700 of the dynamic scene calibration environmentof FIG. 7 includes a number of main elements, each with sub-components.The main elements include the autonomous vehicle 102, a dynamic scenecontrol bridge 710, the motorized turntable system 450, a lightingsystem 760, a target control system 770, a scene surveying system 610,and a power supply system 790.

The autonomous vehicle 102 includes the one or more sensors 180, the oneor more internal computing devices 110, one or more wirelesstransceivers 705 (integrated with communication service 116), and anyother elements illustrated in and discussed with respect to FIG. 1. Thesensors 180 may in some cases include one or more GPRS receivers,Bluetooth® beacon-based positioning receivers, an inertial measurementunit (IMU), one or more cameras, one or more lidar sensors, one or moreradar sensors, one or more sonar sensors, one or more sodar sensors,and/or one or more other sensors discussed herein, which may in somecases be used to identify when the vehicle 102 is not on the platform420, when the vehicle 102 is on the platform 420, when the vehicle 102in a defined position on the platform (e.g., as in FIG. 8B), when thevehicle 102 have begun rotating from a stopped position, and/or when theplatform 420 and/or the vehicle 102 have stopped rotating. The computingdevice 110 of the vehicle 102 (or its sensors 180) may thenautomatically communicate one or more signals or messages through wiredand/or wireless communication interfaces, for instance through thewireless transceiver 705, to the bridge 710, the computing device 110 ofthe vehicle 102 (if the communication is straight from the sensors 180),and/or the computing device 745 of the motorized turntable system 405 toconvey any of these observations/detections/identifications by thesensors 180, which may be used to trigger various actions, such asrotation or stop of rotation of the turntable 405, collection of sensordata or stop of collection of sensor data at the vehicle 102, or somecombination thereof.

The dynamic scene control bridge 710 includes one or more computingdevices 715 and one or more wired and/or wireless transceivers 720. Thedynamic scene control bridge 710 is optional, but can serve as a“middleman” or “router” between the autonomous vehicle 102 and theremaining main elements of the system architecture 700, such as thedynamic scene control bridge 710, the motorized turntable system 450,the lighting system 760, the target control system 770, the scenesurveying system 610, and the power supply system 790. The dynamic scenecontrol bridge 710 can in some cases convert file formats, performmathematical operations such as operation conversions, or otherwiseinterpret instructions or data as necessary so that the vehicle 120 cansuccessfully communicate with other elements. In fact, the dynamic scenecontrol bridge 710 can perform similar conversions, mathematicaloperations, or interpretations in its role as a middleman between anytwo or more devices of the architecture 700.

The motorized turntable system 405 includes a turntable structure 725 aswell as one or more motors, encoders, actuators, and/or gearboxes 730for actuating rotation of the turntable structure 725 while the vehicle102 is on it. The motorized turntable 725 may include a platform 420,which is a surface upon which the vehicle 102 rests during calibration.The platform 420 is rotatable about a base 425 of the motorizedturntable structure 725, with one or more motors 730 that, when actuatedor activated, rotate the platform 420 about the base 425, and which stopthe rotation of the platform 420 about the base 425 when the motors 730are deactivated. For example, the one or more motors 730 may rotate theplatform 420 about the base 425 from a first rotational orientation to asecond rotational orientation, or alternately back to the firstrotational orientation (e.g., if the rotation is a 360 degree rotationor a multiple thereof). A rotational orientation of the platform 420relative to the base 425, or of the vehicle 102 relative to the base425, may alternately be referred to as a rotational position. Themotorized turntable system 405 may include one or more sensors 735, suchas pressure sensors, for example to identify whether or not the vehicle102 is on the turntable structure 725, whether or not the vehicle 102 ispositioned correctly on the turntable structure 725, or how thevehicle's weight is distributed generally or across the platform 420'stop surface (which is in contact with the wheels and/or tires vehicle102) of the turntable structure 725. The sensors 735 may in some casesidentify when the platform 420 has no vehicle 102 on it, when thevehicle 102 is on the platform 420, when the vehicle 102 in a definedposition on the platform (e.g., as in FIG. 8B), when the platform 420and/or the vehicle 102 have begun rotating from a stopped position,and/or when the platform 420 and/or the vehicle 102 have stoppedrotating. The motorized turntable system 405 may then automaticallycommunicate one or more signals or messages through wired and/orwireless communication interfaces to the bridge 710, vehicle 102, and/orthe computing device 745 of the motorized turntable system 405 to conveyany of these observations/detections/identifications by the sensors 735,which may be used to trigger various actions, such as rotation or stopof rotation of the turntable 405, collection of sensor data or stop ofcollection of sensor data at the vehicle 102, or some combinationthereof. The controller 740 may be used to control the actuation of themotors, encoders, actuators, and/or gearbox(es) 730, for example tocontrol a speed or rate or angular velocity of rotation, an angularacceleration (or deceleration) of rotation, a direction of rotation(e.g., clockwise or counterclockwise), or some combination thereof. Themotorized turntable system 405 includes one or more computing devices745 and one or more wired and/or wireless transceivers 750, throughwhich it may interact with the vehicle 102, the dynamic scene controlbridge 710, or any other element in the architecture 700.

The lighting system 760 includes one or more light sources 620 and oneor more motors and/or actuators 762 for activating or turning on each ofthe light sources 620, disabling or turning off each of the lightsources 620, fading or dimming each of the light sources 620,brightening each of the light sources, or moving each of the lightsources 620 with an actuated motor (e.g., to shine on a particulartarget). The lighting system 760 includes one or more computing devices764 and one or more wired and/or wireless transceivers 766, throughwhich it may interact with the vehicle 102, the dynamic scene controlbridge 710, or any other element in the architecture 700.

The target control system 770 includes one or more targets and targetsupport structure 772. The targets may include one or more of any of thetargets 200A, 200B, 200C, 220, and/or 250 illustrated in FIG. 2A-2E, anyother target described herein, any other sensor calibration target, orsome combination thereof. The target support structures may includeeasel-type support structures, tripod-type support structures, or therod-type support structures 410 with wide bases seen in FIG. 4 and FIG.5. The target support structures may include any material discussed withrespect to the substrate 205, such as paper, cardboard, plastic, metal,foam, or some combination thereof. In some cases, certain stands may bemade of a plastic such polyvinyl chloride (PVC) to avoid detection bycertain types of range sensors, such as radar, which detect metal betterthan plastic.

The targets and/or support structures 720 may in some cases bemotorized, and as such, the target control system 770 may include motorsand actuators 774 that it can use to move the targets, for example asrequested by the vehicle 102 to optimize calibration. For example, thetarget support structures may include a robotic arm with ball jointsand/or hinge joints that may be actuated using the motors and actuators774 to translate a target in 3D space and/or to rotate a target aboutany axis. The motors and actuators 773 may alternately only control asingle type of movement for a particular target, for example by enablinga target to rotate about the rod of a stand 410. The target supportstructure 772 may also include wheels or legs, which may be actuated bythe motors 774 to enable the entire target support structure 772 tomove, and with it, the target(s) it supports. The target control system770 includes one or more computing devices 776 and one or more wiredand/or wireless transceivers 778, through which it may interact with thevehicle 102, the dynamic scene control bridge 710, or any other elementin the architecture 700.

The scene surveying system 610 includes a surveying device supportstructure 780, such as a tripod or any other structure discussed withrespect to the target support structure 772, and one or more sensors 782coupled to the support structure 780. The sensors 782 of the scenesurveying system 610, like the sensors 180 of the vehicle 102, mayinclude one or more cameras of any type (e.g., wide-angle lens, fisheyelens), one or more range sensors (e.g., radar, lidar, emdar, laserrangefinder, sonar, sodar), one or more infrared sensors, one or moremicrophones, or some combination thereof. Using these, the scenesurveying system 610 can capture a representation of the entire dynamicscene, including the vehicle 102, allowing determination of distancesbetween the vehicle 102 and various targets. In some cases, either thevehicle 102 or the scene surveying system 610 or both may requestadjustment of lighting through the lighting system 760 and/or adjustmentof target positioning via the target control system 770. The scenesurveying system 610 includes one or more computing devices 784 and oneor more wired and/or wireless transceivers 784, through which it mayinteract with the vehicle 102, the dynamic scene control bridge 710, orany other element in the architecture 700. In some cases, featuretracking and/or image recognition techniques applied using the computingdevice 784 may be used with the a camera and/or the radar, lidar, sonar,sodar, laser rangefinder, and/or other sensors 782 of the scenesurveying system 610 to identify when the platform 420 has no vehicle102 on it, when the vehicle 102 is on the platform 420, when the vehicle102 in a defined position on the platform (e.g., as in FIG. 8B), whenthe platform 420 and/or the vehicle 102 have begun rotating from astopped position, and/or when the platform 420 and/or the vehicle 102have stopped rotating. The scene surveying system 610 may thenautomatically communicate one or more signals or messages through wiredand/or wireless communication interfaces to the bridge 710, vehicle 102,and/or motorized turntable system 405 to convey any of theseobservations/detections/identifications by the scene surveying system610, which may be used to trigger various actions, such as rotation orstop of rotation of the turntable 405, collection of sensor data or stopof collection of sensor data at the vehicle 102, or some combinationthereof. In some cases, the scene surveying system 610 may simply bereferred to as a camera or as another sensor that the scene surveyingsystem 610 includes.

The power supply system 790 may include batteries, generators, or mayplug into an outlet and into the power grid. The power supply system 790may supply power to the various elements and components of the systemarchitecture 700, including at least the dynamic scene control bridge710, the motorized turntable system 450, the lighting system 760, thetarget control system 770, and the scene surveying system 610. The powersupply system 790 may also charge the vehicle 102 before, during, and/orafter calibration, for example if the vehicle 102 is electric or hybrid.The power supply system 790 may also intelligently scale voltage,amperage, and current as appropriate for each element and component ofthe system architecture 700, and to do so it may include a computingsystem 1500 (not pictured). It may also include a wired and/or wirelesstransceiver (not pictured) through which it may interact with thevehicle 102, the dynamic scene control bridge 710, or any other elementin the architecture 700.

The computing devices 110, 715, 745, 764, 776, and 784 may each, atleast in some cases, include at least one computing system 1500 asillustrated in or discussed with respect to FIG. 15, or may include atleast a subset of the components illustrated in FIG. 15 or discussedwith respect to computing system 1500.

FIG. 8A illustrates a top-down view of a turntable with a vehicle guiderail, and a vehicle driving onto the turntable centered relative to thevehicle guide rail.

The motorized turntable system 405 of FIG. 8A includes a guide railing810 on the platform 420 of the turntable 405, particularly on a topsurface of the platform 420, so that the guide railing 810 rotates alongwith the rest of the platform 420 about the base 425 when the turntablemotor 730 is activated. The guide railing 810 includes two substantiallyparallel straight rails 840 with a space in between, illustrated in FIG.8A as parallel vertically-oriented lines, the space between the twostraight rails 840 corresponding to a distance between two wheels and/ortires of the vehicle 102 (e.g., left and right front wheels/tires of thevehicle 102 and/or left and right rear wheels/tires) in such a way thatthe vehicle 102 can straddle the two straight rails 840 with at leastone left wheel and/or tire on a left side of the left straight rail 840and at least one right wheel and/or tire on a right side of the rightstraight rail 840. The rails of the guide railing 810—that is, the twostraight rails 840 and the two slanted rails 845—may extend from the topsurface of the platform 420 in a direction perpendicular to the topsurface of the platform 420.

At one end, these parallel rails end in a “stop” member 850, which mayinclude a rail, a wall, a bump, or an inclined ramp, the stop member 850gently stopping the vehicle 102 from continuing further forward once thevehicle has reached a defined position astride the two straight rails840. That is, each stop member 850 may include a vertical wall or railthat extends in a direction perpendicular to the top surface of theplatform 420, or an inclined ramp or bump that gradually increases inheight (height being a direction perpendicular to the surface of theplatform 420) along a direction in which the vehicle 102 is driving ontoor along the platform 420 to form an incline that, if the vehicle 102were to drive at least partially onto the incline, would push thevehicle 102 backward. The incline may be straight, have a convex curve,or have a concave curve. The incline may be part of a bump that goes upin height and then back down, such as a speed bump. That is, regardlessof the form of the stop members 850, if the vehicle 102 reaches thedefined position at which it is pictured in FIG. 8B and attempts tocontinue driving toward the one or more stop members 850, the vehicle102 will come into contact with one or more stop members 850, which willpush the vehicle back into the defined position, either gently using anincline or more forcefully using a wall or rail. The vehicle 102 maydrive forwards onto the platform 420 and toward the stop members 850 asillustrated in FIG. 8A (i.e., with the front side of the vehicle 102facing the direction of driving and the stop members 850), or mayreverse onto the platform 420 and toward the stop members 850 (e.g.,drive with the rear side of the vehicle 102 facing the direction ofdriving and the stop members 850). A wall or rail included in a stopmember 850 may in some cases be padded using foam, silicone, rubber, oranother soft or bouncy material that cushions the force applied by thestop member 850 on the vehicle 102 and the force applied by the vehicle102 on the stop member 850. The stop members 850 of the guide railing810 of FIG. 8A are illustrated as two relatively short horizontal lines(perpendicular to the parallel vertical guide rails) at the topendpoints (i.e., posterior ends) of the parallel straight rails 840 ofthe guide railing 810. In some cases, one of the two stop members 850may be missing, as stopping one tire/wheel from progressing may besufficient to stop the vehicle 102 from moving beyond the stop member850. While the two stop members 850 are illustrated as a separate memberfor each of the two straight rails 840, in some cases (not pictured), asingle longer stop member 850 can be used that bridges the gap betweenthe straight rails 840 and comes into contact with the posterior ends ofboth straight rails 840 in a direction that is perpendicular to bothstraight rails, extending beyond the posterior ends of one or bothstraight rails 840.

The other end (i.e., the anterior end) of each of the two parallelstraight rails 840 ends in a slanted rail 845. Relative to the twoparallel straight rails 840, the two slanted rails 845 slant towards oneanother. The slanted rails 845 are generally not parallel to one anotheror to the straight rails 840, though may include portions that areparallel to one or both (e.g., when the slanted rails 845 are curved).That is, the leftmost slanted rail 845 slants to the right as itproceeds in further downward (i.e., in a more anterior direction), whilethe rightmost slanted rail 845 slants to the left as it proceeds infurther downward (i.e., in a more anterior direction). In the guidancerail 810 illustrated in FIG. 8A, the two slanted rails 845 are straightas well, and approach a single point (i.e., a vertex) at which theymeet/converge. That point at which the slanted rails 845 meet/convergemay be along a center line 805/815 that is parallel to the two straightrails 840 and that is centered (i.e., equidistant) between the twostraight rails 840. Accordingly, the two slanted rails 845, together,form a “V” shape. In some cases (not illustrated), the two slanted rails845 need not meet at the point they both approach, thereby remainingseparate and effectively removing a bottom portion/fraction of the “V”shape that includes the vertex. In some cases (not illustrated), thevertex of the “V,” the vertices between the slanted rails 845 and thestraight rails 840, and/or any other vertices formed by the slantedrails 845 and/or the straight rails 840 may be smoothed out so as not topresent a risk of puncturing or otherwise damaging a tire or wheel ofthe vehicle 102. In some cases (not illustrated), the two slanted rails845 are curved rather than straight, for example forming a “U” shaperather than a “V” shape, or at least a top portion of a “U” shape. Thatis, each of the slanted rails 845 may include one or more curves thateach form a portion of a circle or a portion of an ellipse, each curveof which may for example be concave facing inward (i.e., facing thespace between the two parallel straight rails 840) and convex facingoutward (i.e., facing the base 425 of the turntable and the rest of thecalibration environment) or vice versa. In some cases, each of theslanted rails 840 may include one or more straight portions and one ormore curved portions, and may for example form a curve similar to acurve formed by graphing a tangent function (e.g., ƒ*tan(x) where ƒ is apositive or negative constant) in a 2-dimensional X-Y space.

For structural integrity, the guide railing 810 may also includeoptional braces (not pictured) connecting the left and right straightrails 840 to one another through the space in between the two straightrails 840. Similar braces may exist between the two slanted rails 845,connecting the two slanted rails 845. Similar braces may exist betweenthe two stop members 850, connecting the two stop members 850. Thebraces may include braces that are perpendicular to the parallelvertical guide rails or diagonal relative to the parallel vertical guiderails (e.g., forming one or more cross (“X”) structures). The braces mayinclude braces that are perpendicular to at least a portion of one orboth of the slanted rails 845 or diagonal relative to at least a portionof one or both of the slanted rails 845.

The vehicle 102 of FIG. 8A approaches the motorized turntable system 405along a centered path 805, represented by dashed line arrow, that iscentered with respect to the turntable 405 and the guide railing 810. Asa result, the vehicle 102 will successfully drive onto the center of theplatform 420 of the motorized turntable 405, eventually reaching adefined position in which the wheels and/or tires of the vehicle 102straddle (i.e., are positioned astride) the two straight rails 840 andabut the stop members 850, the vehicle 102 illustrated in this definedposition in FIG. 8B. The defined position represents the position thatthe guide railing 810 guides the vehicle 102 into.

In some cases (not pictured), the guide railing 810 may intentionally bepositioned off-center (horizontally and/or vertically) relative to thecenter of the platform 420 of the motorized turntable system 405. Forexample, the guide railing 810 may be moved further forward relative tothe platform 420 to accommodate and/or center a larger or longer vehicle102 than the illustrated sedan-style vehicle 102 (e.g., a van, truck, orlimousine) on the platform 420, or further backward relative to theplatform 420 to accommodate and/or center a smaller or shorter vehicle102 than the illustrated sedan-style vehicle 102 (e.g., a compactautomobile, buggy, or all-terrain vehicle) on the platform 420. Theguide railing 810 may be moved left or right as well, as it may bedesirable for the vehicle 102 to move translationally in space duringcertain types of sensor calibrations rather than to just rotate about anaxis that passes through the vehicle, effectively rotating in smallcircular “laps” within a radius of the platform 420 but with the benefitof the precision motor control of the turntable 405 (as opposed to usingthe vehicle 102's engine for propulsion) and of not having to consumefuel or battery power of the vehicle 102. In some cases, the guiderailing 810 may be adjusted dynamically. To achieve such adjustabilityin a dynamic fashion, the guide railing 810 itself may be attached tothe platform 420 via additional rail tracks (not shown) along theplatform 420 that are optionally recessed into the platform, the railtracks along the platform 420 allowing the guide railing 810 to be slidalong the rail tracks relative to the surface of the platform 420, withone or more latches and/or magnets and/or screws used to lock the guiderailing 810 in a particular position along the rail tracks along theplatform 420. Movement of the guide railing 810 along such rail tracksalong the platform 420 may also be activated and/or deactivated usingone or more motors that may be controlled by the motorized turntablesystem 405 and/or by the vehicle 102 and/or otherwise as discussed withrespect to actuation of the motors, encoders, actuators, and/orgearbox(es) 730.

FIG. 8B illustrates a vehicle having successfully driven onto theturntable centered along the vehicle guide rail, the vehicle thuscentered with respect to the turntable.

The vehicle 102 in FIG. 8B, which is illustrated as transparent with adashed line outline and grey-filled wheels/tires, has driven onto themotorized turntable system 405. Through the transparent vehicle outline,we can see that the wheels and tires of the vehicle 102 are adjacent tothe outsides of the parallel rails of the guide railing 810, and thestop member 850 has stopped the vehicle 102 at the defined position,which in FIG. 8B is in the center of the platform of the turntable 405,but may alternately be positioned further forward, backward, and/or toone of the sides of the platform 420 as discussed with respect to FIG.8A.

While the two straight rails 840 illustrated in FIGS. 8A-8D are onlylong enough for two wheels and/or tires of the vehicle 102 (either thefront left and right wheels/tires or the rear left and rightwheels/tires) to stand astride the two straight rails 840 while thevehicle 102 is in the defined position, in some cases the two straightrails 840 may be long enough so that all four or more wheels and/ortires of a vehicle 102 stand astride the two straight rails 840 whilethe vehicle 102 is in the defined position. That is, if the vehicle is asix-wheeler or eight-wheeler or ten-wheeler truck or other long vehicle102, all of the left and right pairs of wheels, or any subset of thepairs of wheels, can stand astride the two straight rails 840 while thevehicle 102 is in the defined position. FIG. 8C illustrates a top-downview of a turntable with a vehicle guide rail, and a vehicle drivingonto the turntable off-center relative to the vehicle guide rail.

The vehicle 102 of FIG. 8C approaches the platform of the motorizedturntable system 405 along an un-centered path 820, represented bydashed line arrow, which is off-center with respect to the turntable 405and the guide railing 810, as is visible when compared to the centerline 815, represented by a dotted line. The center line 815 is a linerepresenting the centered path 805 from FIG. 8A and thus representscenter with respect to the platform of the turntable 405 and the guiderailing 810 and the defined position of the vehicle 102 in FIG. 8B. Thecenter line 815 is illustrated in FIG. 8C to visibly highlight that theun-centered path 820 is off-center with respect to the platform of theturntable 405 and the guide railing 810 and the defined position of thevehicle 102 in FIG. 8B. As a result, the vehicle 102 will drive onto theplatform of the motorized turntable 405 off-center with respect to theplatform of the turntable 405 and the guide railing 810 and the definedposition of the vehicle 102 in FIG. 8B, and will approach the guiderailing 810 off-center off-center with respect to the platform of theturntable 405 and the guide railing 810 and the defined position of thevehicle 102 in FIG. 8B as visible in FIG. 8D.

FIG. 8D illustrates a vehicle having driven partially onto the turntablewhile off-center relative to the vehicle guide rail, the vehicle guiderail guiding the vehicle's path to center the vehicle with respect tothe turntable.

The vehicle 102 in FIG. 8D, which is illustrated as transparent with adashed line outline and grey wheels/tires, has driven onto the motorizedturntable system 405 off-center. Through the transparent vehicleoutline, we can see that one of the wheels/tires of the vehicle 102 iscontacting the slanted rails 845 (e.g., the “V” or “U” portion) of theguide railing 810 at a guidance point 830. If the vehicle 102 keepsdriving forward relative to the position it is illustrated in in FIG.8D, the the slanted rail 840 will push back against the vehicle 102 atthe guidance point 830 (i.e., the contact point), as will furthercontact points along the same slanted rail 840 of the guide railing 810as the vehicle 102 continues to progress forward (diagonally) pressingagainst the same slanted rail 840, and thus the slanted rail 840 of theguide railing 810 will automatically guide the vehicle 102 to the righttoward the centered/defined position until the vehicle 102 reaches thesubstantially parallel vertical rail portions of the guide railing 810,after which the vehicle 102 can drive straight without encumbrance.Thus, through application of forward torque/force by the vehicle 102 onthe slanted rail 845 at the guidance point 830, and through reciprocalforce pushing back diagonally on the vehicle 102 from the guidance point830 of the slanted rail 420 of the guide railing 810, eventually, theslanted rail 420 of the guide railing 810 will assist the vehicle 102into a centered position with respect to the guide railing 810 and theplatform 420 of the turntable 405, and the vehicle 102 will ultimatelyreach the defined position that the vehicle 102 is illustrated havingreached in FIG. 8B after moving forward along the straight rails 840.While FIGS. 8C and 8D illustrate the effect of the guide railing 810 oncentering the vehicle 810 along the platform 420 of the turntable system405 when the vehicle 102 is approaching off-center to the left relativeto the guide railing 810 by pushing the vehicle 102 to the right, asimilar effect occurs when the vehicle 102 is approaching off-center tothe right relative to the guide railing 810 (not illustrated) by pushingthe vehicle 102 to the left. Thus, the guide railing 810 allows for easyand consistent positioning of the vehicle 102 at a defined positionalong the platform 420 of the turntable 405.

FIG. 9 is a flow diagram illustrating operation of a calibrationenvironment.

At step 905, a high resolution map of calibration environment isgenerated. This may be performed using the scene surveying system 610,for example.

At step 910, all sensors 180 on the vehicle 102 are run in thecalibration environment, for example at different rotation positions ofthe vehicle 102, which is rotated using motorized turntable 405. At step915, the vehicle 102 generates a calibration scene based on its sensors180, based on (a) synchronized sensor data, (b) initial calibrationinformation, (c) vehicle pose information, and (d) target locations.

At step 915, the calibration systems in the vehicle read the calibrationscene and: (a) detect targets in each sensor frame, (b) associatedetected targets, (c) generate residuals, (d) solve calibrationoptimization problem, (e) validate calibration optimization solution,and (f) output calibration results. At step 925, the calibration resultsare tested against acceptable bounds and checked for numericalsensitivity. Successful calibration measurements are stored and logged,along with a minimal subset of data needed to reproduce them

FIG. 10 is a flow diagram illustrating operations for intrinsiccalibration of a vehicle sensor using a dynamic scene.

At step 1005, a vehicle 102 is rotated into a plurality of vehiclepositions over a course of a calibration time period using a motorizedturntable 405. The vehicle 102 and motorized turntable 405 are locatedin a calibration environment. At step 1010, the vehicle 102 captures aplurality of sensor capture datasets via a sensor coupled to the vehicleover the course of the calibration time period by capturing at least oneof the plurality of sensor capture datasets while the vehicle is at eachof the plurality of vehicle positions.

At step 1015, an internal computing system 110 of the vehicle 102receives the plurality of sensor capture datasets from the sensorcoupled to the vehicle over a course of a calibration time period. Atstep 1020, the internal computing system 110 of the vehicle 102identifies, in the plurality of sensor capture datasets, one or morerepresentations of (at least portions of) the calibration environmentthat include representations of a plurality of sensor targets. Theplurality of sensor targets are located at known (i.e., previouslystored) positions in the calibration environment. At steps 1025-1030,the sensor is calibrated based on the representations of a plurality ofsensor targets identified in the plurality of sensor capture datasets.

More specifically, at step 1025, the internal computing system 110 ofthe vehicle 102 identifies positions of the representations of theplurality of sensor targets within the one or more representations of(at least portions of) the calibration environment. If the sensor beingcalibrated is a camera, and the one or more representations of (portionsof) the calibration environment are images, then the representations ofthe sensor targets may be areas within the one or more images comprisedof multiple pixels, which the computing system 110 of the vehicle 102can identify within the one or more images by generating high-contrastversions of the one or more images (i.e., “edge” images) that areoptionally filtered to emphasize edges within the image, and byidentifying features within the image comprised of one or more of thoseedges, the features recognizable as portions of the target. For example,the vertices and/or boxes in the checkerboard pattern 210A or the ArUcopattern 210B, curves or vertices in the crosshair pattern 210C, thecircular ring marking patterns 230, or combinations thereof, may each bevisually recognized as features in this way. Similarly, if the sensorbeing calibrated is a radar sensor, the radar sensor may recognize thetrihedral shape 215 of the target 220 as a feature due to its reflectivepattern that results in a high radar cross section (RCS) return.Similarly, if the sensor being calibrated is a lidar sensor, the lidarsensor may recognize the surface of the substrate 205 of the target 250and the apertures 225/240 within the substrate 205 of the target 250,which may be recognized as a feature due to the sharp changes inrange/depth at the aperture.

At step 1030, the internal computing system 110 of the vehicle 102generates a transformation that maps (1) the positions of therepresentations of the plurality of sensor targets within one or morerepresentations of (portions of) the calibration environment to (2) theknown positions of the plurality of sensor targets within thecalibration environment. Other information about the plurality of sensortargets, such as information storing visual patterns or aperturepatterns of the sensor targets, may also be used to generate thetransformation. For example, if the sensor being calibrated is a camera,and the computing device 110 knows that an image should have acheckerboard pattern 210A of a sensor target 200A, and recognizes awarped or distorted variant of the checkerboard pattern 210A (e.g.,because the camera includes a fisheye lens or wide-angle lens), then thecomputing device 110 may use its knowledge of the way that thecheckerboard should look, such as how far the vertices are from eachother, that they should form squares, and that the squares are arrangedin a grid pattern—to generate a transformation that undoes thedistortion caused by the camera, thereby mapping the vertices detectedin the image to real-world positions, at least relative to one another.In other words, the transformation includes one or more projectivetransformations of various 2-D image coordinates of sensor targetfeatures into 3-D coordinates in the real world and optionally back into2-D image coordinates that have been corrected to remove distortionand/or other sensor issues.

Because the computing device 110 knows ahead of time exactly where thesensor targets are in the calibration environment, the transformationmay also map the positions of the vertices in the image (and thereforethe positions of the representations of the sensor targets in therepresentation of the calibration environment) to real-world positionsin the calibration environment. The transformation(s) that are generatedduring intrinsic sensor calibration at step 1030 can include one or moretypes of transformations, including translations, stretching, squeezing,rotations, shearing, reflections, perspective distortion, distortion,orthogonal projection, perspective projection, curvature mapping,surface mapping, inversions, linear transformations, affinetransformations, The translational and rotational transformations mayinclude modifications to position, angle, roll, pitch, yaw, orcombinations thereof. In some cases, specific distortions may beperformed or undone, for example by removing distortion caused by use ofa specific type of lens in a camera or other sensor, such as awide-angle lens or a fisheye lens or a macro lens.

Step 1030 may be followed by step 1005 and/or by step 1010 ifcalibration is not yet complete, leading to gathering of more sensorcapture datasets and further refinement of the transformation generatedat step 1030. Step 1030 may alternately be followed by step 1045.

The previously stored information about the plurality of sensor targetsmay be from a high-definition map generated as in step 905 of FIG. 9,may be from a second sensor on the vehicle, or may simply be based on aprior understanding of the sensor targets. For example, the internalcomputing system 110 of the vehicle 102 understands what a checkerboardpattern 210A is and that the grid it forms ought to look include a gridof parallel and perpendicular lines under normal conditions. Because ofthis, the internal computing system 110 understands that if therepresentation it received from the sensor (camera) of a target with acheckerboard pattern 210A forms a grid warped or distorted by awide-angle lens or fisheye lens, this difference (step 1030) can becorrected by the internal computing system 110 by correctivelydistorting or warping the image captured by the sensor (camera) by toreverse the warping or distortion in the representation until thecorrected checkerboard looks like it should. This corrective warping ordistortion is the correction generated in step 1035. The correction mayalso include a translation along X, Y, or Z dimensions, a rotation alongany axis, a warp or distortion filter, a different type of filter, orsome combination thereof

Steps 1045-1060 concern operations that occur after calibration iscomplete (i.e., post-calibration operations). At step 1045, the sensorof the vehicle captures a post-calibration sensor capture dataset afterthe calibration time period, after generating the transformation, andwhile the vehicle is in a second position that is not in the calibrationenvironment. At step 1050, the computing device 110 of the vehicle 102identifies a representation of an object within a representation of ascene identified within the post-calibration sensor capture dataset. Atstep 1055, the computing device 110 of the vehicle 102 identifies aposition of the representation of the object within the representationof the scene. At step 1060, the computing device 110 of the vehicle 102identifies a position of the object relative to the second position ofthe vehicle by applying the transformation to the position of therepresentation of the object within the representation of the scene.

Note that capture of data by the sensors 180 of the vehicle 102 mayoccur in parallel with calibration of the sensors 180 of the vehicle102. While an initial correction is generated at step 1035, the vehicle102 may continue to rotate, and its sensors 180 may continue to capturemore sensor data, hence the dashed lines extending back up to steps 1005and 1010 from step 1035. When step 1035 is reached a second, third, orN^(th) time (where N is any integer over 1), the correction generatedthe first time step 1035 was reached may be updated, revised, and/orre-generated based on the newly captured sensor data when step 1035 isreached again. Thus, the correction becomes more accurate as calibrationcontinues.

For some additional context on intrinsic calibration: LIDAR intrinsicproperties may include elevation, azimuth, and intensity. Cameraintrinsic properties may be given as matrices based on cameraregion/bin, and may track projection, distortion, and rectification. Allsensors' intrinsic properties (including LIDAR and camera) may includeposition in X, Y, and/or Z dimensions, as well as roll, pitch, and/oryaw.

FIG. 11 is a flow diagram illustrating operations for extrinsiccalibration of two sensors in relation to each other using a dynamicscene.

At step 1105, a vehicle 102 is rotated into a plurality of vehiclepositions over a course of a calibration time period using a motorizedturntable 405. At step 1110, the vehicle 102 captures a first pluralityof sensor capture datasets via a first sensor coupled to the vehicleover the course of the calibration time period by capturing at least oneof the first plurality of sensor capture datasets while the vehicle isat each of the plurality of vehicle positions. At step 1115, the vehicle102 captures a second plurality of sensor capture datasets via a secondsensor coupled to the vehicle over the course of the calibration timeperiod by capturing at least one of the first plurality of sensorcapture datasets while the vehicle is at each of the plurality ofvehicle positions. Either of steps 1110 and 1115 can occur first, orthey can occur at least partially in parallel.

At step 1120, the internal computing system 110 of the vehicle 102receives the first plurality of sensor capture datasets from the firstsensor and the second plurality of sensor capture datasets from thesecond sensor. At step 1125, the internal computing system 110 of thevehicle 102 identifies, in the first plurality of sensor capturedatasets, representations of a first plurality of sensor targetfeatures, the first plurality of sensor target features detectable bythe first sensor due to a type of the first plurality of sensor targetfeatures being detectable by sensors of a type of the first sensor. Atstep 1130, the internal computing system 110 of the vehicle 102identifies, in the second plurality of sensor capture datasets,representations of a second plurality of sensor target features, thesecond plurality of sensor target features detectable by the secondsensor due to a type of the second plurality of sensor target featuresbeing detectable by sensors of a type of the second sensor. Either ofsteps 1125 and 1130 can occur first, or they can occur at leastpartially in parallel.

The first plurality of sensor target features and the second pluralityof sensor target features may be on the same targets; for example, ifthe first sensor is a camera, and the second sensor is a LIDAR sensor,and plurality of sensor targets are the combined extrinsic calibrationtargets 250 of FIGS. 2E and 5, then the first plurality of sensor targetfeatures may be the visual markings (rings) 230 detectable by thecamera, while the second plurality of sensor target features are theapertures 225 detectable by the LIDAR. Alternately, the plurality ofsensor target features may be on different targets; for example, thefirst sensor may be a radar sensor and the first plurality of sensortarget features may be the trihedral radar calibration targets 220,while the second sensor may be any other sensor (e.g., camera, lidar)and the second plurality of sensor target features may be, for example,the visual markings (rings) 230 or apertures 225 of the combinedextrinsic calibration targets 250, or a pattern 210 of a cameraintrinsic target such as the targets 200A-C, or any other targetdescribed herein.

At step 1135, the internal computing system 110 of the vehicle 102compares the relative positioning of the representations of the firstplurality of sensor target features and the representations of thesecond plurality of sensor target features to known relative positioningof the first plurality of sensor target features and the secondplurality of sensor target features. In some cases, the relativepositioning may be determined based on comparison of a position of aparticular point in one representation, such as the center, to aparticular point in the another representation to which it is beingcompared, such as the center. Points that can be used instead of thecenter may include or the highest point, lowest point, leftmost point,rightmost point, a point that is centered along one axis but notanother, a point at the widest portion of the representation, a point atthe narrowest portion of the representation, a point at a particularedge or vertex, or some combination thereof. At step 1140, the internalcomputing system 110 of the vehicle 102 generates a transformation basedon the comparison, such that the transformation aligns a first locationidentified by the first sensor and a second location identified by thesecond sensor.

As a first example, the first sensor may be a camera and the secondsensor may be a LIDAR sensor, and the first plurality of sensor targetfeatures and the second plurality of sensor target features may both befeatures of the combined extrinsic calibration targets 250 of FIGS. 2Eand 5 such that the first plurality of sensor target features are thevisual markings (rings) 230 detectable by the camera and the secondplurality of sensor target features are the apertures 225 detectable bythe LIDAR. In such a case, the internal computing system 110 of thevehicle 102 identifies a center of a particular aperture 225 based onthe LIDAR data, and identifies a center of a ring 230 based on thecamera data, compares these at step 1135 and identifies a relativedistance between the two locations based on the internal computingsystem 110's current geographic understanding of the calibrationenvironment. Because the internal computing system 110 understands thatthese two points should represent the same location in the real world(i.e., their relative positioning indicates no distance between them),the internal computing system 110 generates a transformation—which mayinclude, for example, a translation along X, Y, and/or Z dimensions, arotation along any axis, a warp or distortion filter, or somecombination thereof—that aligns these location points. That is, thetransformation translates (1) a mapping of a point from the one sensor'scapture data set to a real world position into (2) a mapping of a pointfrom the other sensor's capture data set to the same real worldposition. While, with just a pair or two of such points, there may bemultiple possible transformations that can perform this alignment, theinternal computing system 110 can generate a transformation that worksconsistently for an increasing number of pairs such sets of points—forexample, for each aperture 225 and ring 230 combinations of a target250, and for each target 250 in the calibration environment. As thenumber of pairs increases, the number of possible transformations thatcan successfully align these. Different sensors may map the world aroundthem differently; for example, if the camera includes a wide-angle lenswhile the other sensor (e.g., LIDAR) does not include an analogousdistortion effect, the transformation may include some radial movementor other compensation for distortion.

As a second example, the first sensor may be a radar sensor and thesecond sensor may be a LIDAR sensor, and the first plurality of sensortarget features may be trihedral radar calibration targets 220 while thesecond plurality of sensor target features may be apertures 225 of acombined target 250 or the planar boundaries of a substrate 205 of acamera target 200, each of which is a known distance away from thenearest trihedral radar calibration targets 220. In such a case, theinternal computing system 110 of the vehicle 102 identifies a locationof the trihedral radar calibration targets 220 based on radar sensordata and a location of the LIDAR target feature based on LIDAR sensordata, compares these at step 1135 and identifies a relative distancebetween the two locations based on the internal computing system 110'scurrent geographic understanding of the calibration environment. Becausethe internal computing system 110 understands that these two pointsshould be a known distance away in a particular direction at aparticular angle in the real world, the internal computing system 110generates a transformation—which may include, for example, a translationalong X, Y, and/or Z dimensions, a rotation along any axis, a warp ordistortion filter, or some combination thereof—that aligns theselocation points to match the same known distance away in the particulardirection at the particular angle as in the real world. While initiallythere may be multiple possible transformation that can perform this, theinternal computing system 110 can generate a transformation that worksconsistently for multiple such sets of points—for example, for eachtrihedral radar calibration target 220 and each nearby LIDAR targetfeature pair in the calibration environment.

At step 1145, the internal computing system 110 of the vehicle 102receives, from the first sensor and second sensor, post-calibrationsensor capture datasets captured by the first sensor and second sensorafter the calibration time period. At step 1150, the internal computingsystem 110 of the vehicle 102 applies the transformation generated instep 1140 to one or both of the post-calibration sensor capturedatasets. For example, a representation of a particular object can beidentified in a post-calibration sensor capture dataset captured by onesensor after calibration, and the transformation can be applied to findthe same object within another post-calibration sensor capture datasetcaptured by another sensor after calibration. A real-world position ofthe same object may be found relative to the vehicle 102 based onintrinsic calibration of at least one of the two sensors and/or based onthe transformation. In some cases, a representation of an entirespace—that is, a three-dimensional volume—in one post-calibration sensorcapture dataset captured by one sensor after calibration may then beidentified in another post-calibration sensor capture dataset capturedby another sensor by applying the transformation to multiple pointswithin the space. Important points, such as vertices (e.g., corners of aroom), edges (e.g., edges of a room), or other features may be selectedas at least some of these points. With two aligned representations of a3-D space, objects can be identified around the vehicle that might nototherwise be. For example, a pedestrian wearing all black might notvisually stand out against (e.g., contrast against) a background of anasphalt road at night, but a RADAR or LIDAR might easily identify thepedestrian, and the transformation will still allow the computer 110 ofthe vehicle 102 to understand where that pedestrian is in its camerafootage, allowing the vehicle to pay close attention to visual cues fromthe pedestrian that the RADAR or LIDAR might not catch or understand,such as presence or lack of a pet or small child accompanying thepedestrian. Developing the vehicle's understanding of its surroundingsby aligning real-world (and relative) mappings of the inputs it receivesfrom its sensors can save lives in the field of autonomous vehicles byallowing the best aspects of multiple sensors to complement one anotherto develop a comprehensive view of the vehicle's surroundings. No sensoris perfect at detecting everything—range sensors can see range/depth butnot color or brightness, and can have trouble seeing small orfast-moving objects—while cameras can see color and brightness andvisual features but can have trouble with depth perception. Thus, eachsensor has its strengths, and the alignment made possible by theextrinsic calibration processes discussed in FIG. 11 can allow the bestaspects of each sensor (the “pros” of each sensor) to complement eachother and compensate for the downsides of each sensor (the “cons” ofeach sensor). Note that capture of data by the sensors 180 of thevehicle 102 may occur in parallel with calibration of the sensors 180 ofthe vehicle 102. While an initial transformation is generated at step1140, the vehicle 112 may continue to rotate, and its sensors 180 maycontinue to capture more sensor data, hence the dashed lines extendingback up to steps 1105 and 1110 and 1115 from step 1140. When step 1140is reached a second, third, or N^(th) time (where N is any integer over1), the transformation generated the first time step 1140 was reachedmay be updated, revised, and/or re-generated based on the newly capturedsensor data when step 1140 is reached again. Thus, the transformationbecomes more accurate as calibration continues.

For some additional context on extrinsic calibration, all sensors'extrinsic properties may include relative positions in X, Y, and/or Zdimensions, as well as roll, pitch, and/or yaw. Target and vehiclelocations are ground truthed via the mapping system discussed in step910 and further as discussed with respect to the transformation of step1030 of FIG. 10. Sensors of the vehicle 102 and scene surveying system610 record the scene and each target is detected using a targetdetection method specific to that sensor and target pair. The measuredtarget location is compared against the mapped target location to derivethe total extrinsic sensor error:

Extr_(sensor)(R,t)=Σ_(target) ∥RC _(target) +t−D _(target)∥²

Where C_(target) is the measured location of the target and D_(target)is the mapped location of the target. We can collect the intrinsicsensor calibration data (as in FIG. 10) in a similar way, at each frameof recorded data the targets are detected and intrinsics are collected.These intrinsic sensor calibration data (as in FIG. 10) might be themeasured distance between pixel coordinates and the lines on a target,or lidar point coordinates and detected planar sides of a target. Thetotal error for a single sensor can be summarized as:

Extrintr(R,t,α)_(sensor)=Intr_(sensor)(α)+γ_(sensor)Extr_(sensor)(R,t)

The weight γ_(sensor) determines the contribution of that sensor'sextrinsic parameter. By collecting the Extrintr for every sensor wedefine a global cost function that describes all intrinsics andextrinsics in the system. We can minimize the total expected error bytoggling the calibration parameters for each sensor [R,t,a] via a convexoptimization algorithm. The output of the sensor extrinsic calibrationsmay be a pair of rotation and translation matrices on a per sensor basiswith respect to the origin of the 3D space (e.g., as identified viaLIDAR).

After the calibration parameters are solved for, tests for the numericalsensitivity of the solution can be performed. This may include, forexample, verifying the Jacobian of the solution is near zero in alldirections and that the covariance of each parameter is reasonably small(e.g., below a threshold). More sophisticated routines that test forsensitivity to targets and constraints may also be performed.

FIG. 12 is a flow diagram illustrating operations for interactionsbetween the vehicle and the turntable.

At optional step 1205, the turntable 405, vehicle 102, or surveyingsystem 610 identifies that the vehicle 102 is positioned on the platformof the motorized turntable. This may be identified using pressuresensors 735 of the turntable 405, a GNSS or triangulation-basedpositioning receiver of the vehicle 102 compared to a known location ofthe turntable 405, image/lidar data captured by the surveying system 610indicating that the vehicle 102 is positioned on motorized turntable405, or some combination thereof. In some cases, the pressure sensors735 may be positioned under or beside the guide railing, for exampleclose behind the “stop” wall or incline, to ensure that the vehicle willapply pressure to them. In other cases, the entire turntable isreceptive as a pressure sensor. In any case, this information iscommunicated to the dynamic scene control bridge 710 and/or thecomputing device 745 of the turntable system 405, either within theturntable itself (if via sensors 735) or via the relevant transceiver(s)of FIG. 7. In some cases, either sensor data capture by the sensors ofthe vehicle 102 or rotation of the platform 420 of the motorizedturntable 405 may automatically begin once the pressure sensors identifythat the vehicle 102 is on the platform 420 and/or once sensors identifythat the vehicle 102 has stopped moving (e.g., IMU of the vehicle 102,regional pressure sensors of regions of the turntable platform 420surface, scene surveying system 610 camera, or some combinationthereof). Rotation of the platform 420 about the base 425 may occurfirst before sensor data capture if, for example, calibration ispreviously designated to start with the vehicle 102 rotated to aparticular rotation orientation or rotation position that is not thesame as the rotation orientation or rotation position that the vehicle102 is in when it drives (or is otherwise placed) onto the platform 420.

In some cases, the rotation of the platform 420 of the turntable 405about the base 425 via the motors 730 can manually be triggered insteadof being based on, and automatically triggered by, detection of thevehicle at step 1205, for example via an input received at the dynamicscene control bridge 710 and/or the computing device 745 of theturntable system 405 from a wired or wireless interface that itselfreceives an input from a human being, the wired or wireless interfacebeing for example a keyboard or touchscreen or mouse or remote controlcommunicating in a wired or wireless fashion with the dynamic scenecontrol bridge 710 and/or the computing device 745 of the turntablesystem 405.

At step 1210, one or more motors 730 of the motorized turntable 405 areactivated to rotate the platform 420 of the motorized turntable 405about the base 425 of the motorized turntable 405 (and therefore vehicle102 on top of the platform 420 as well) from a first rotationorientation to a second rotation orientation in response to detectionthat the vehicle is on the turntable. The one or more motors 730 may bedeactivated, causing the platform of the motorized turntable 405 (andtherefore vehicle 102 on top of the platform 420 as well) to stoprotating about the base 425 with the platform 420 in the secondorientation at the stop of rotation. The term “rotation orientation” maybe used to refer to an angle, or angular orientation, or angularposition. Other terms may be used in place of the term “rotationposition,” such as “angle,” “angular position,” “angular orientation,”“position,” or “orientation.” The first rotation orientation and thesecond rotation orientation may be a predetermined angle away from eachother, for example N degrees, where N is 1, 2, 3, 4, 5, 6, 7, 8, 9, 10,11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28,29, 30, or some amount in between any two of these numbers. The firstrotation orientation and the second rotation orientation may be an angleaway from each other that is determined by the internal computing system110 of the vehicle 102, or by the dynamic scene control bridge 710, orby the computing device 745 of the turntable system 405, or by somecombination thereof, based on which angle would likely be mostefficient, comprehensive, or an optimal balance thereof, in completingcalibration of the entire fields of view (FOV) of the sensors 180 of thevehicle 102.

At step 1215, the vehicle 102 uses its IMU (or other rotation detectiondevice) to check whether the vehicle 102 (and therefore the platform420) is still rotating. As the IMU is a collection of accelerometersand/or gyroscopes and/or other motion or rotation detection devices, thevehicle 102 can alternately separately use accelerometers and/orgyroscopes and/or other motion or rotation detection devices that areamong the vehicle 102's sensors 180 to determine this. Alternately, theturntable 405 may use one or more motion sensors of its own (e.g.,accelerometer, gyroscope, IMU, or any other motion sensor discussedherein) to identify whether the platform 420 of the turntable 405 isstill rotating about the base 425. Alternately still, the scenesurveying system 610 may use one or more cameras to visually identifywhether the platform of the turntable 405 and/or the vehicle 102 isstill rotating. In some cases, the device that detects that the vehicle102 and/or the platform 420 of the turntable 405 has stopped rotatingrelative to the base 425 (the vehicle computing system 110, thecomputing device 745 of the turntable 405, the scene surveying system610, and/or the dynamic scene control bridge 710) can send a signalidentifying the detected stop in rotation to any of the vehiclecomputing system 110, the computing device 745 of the turntable, thescene surveying system 610, or the dynamic scene control bridge 710.

If, at step 1220, the vehicle 102 or turntable 405 or scene surveyingsystem 610 determines that the rotation has stopped, step 1225 followsstep 1220. Otherwise, step 1215 follows step 1220.

In addition, we may use the vehicle 102's other sensors 180, such as oneor more cameras, radar sensors, lidar sensors, sonar sensors, and/orsodar sensors instead of or in addition to the IMU, accelerometers,gyroscopes, and/or motion/rotation detection devices to identify whenthe vehicle 102 (and thus the platform 420) is still rotating relativeto the base 425 or not. With all of these sensors, rotation may beidentified based on whether regions of the calibration environment thatshould be motionless—walls, the floor, the ceiling, targets that havenot been configured and/or commanded to move, light sources 620, thescene surveying system 610—are changing location between sensor captures(indicating that the vehicle is rotating and/or in motion) or arestationary between sensor captures (indicating that the vehicle isstationary).

At step 1225, the vehicle captures sensor data using one or more of itssensors while the vehicle 102 is at the second position. If, at step1230, the internal computing device 110 of the vehicle 102 determinesthat it has finished capturing sensor data while vehicle is at thesecond rotational orientation/position, then step 1235 follows step1230, and optionally, the vehicle computing system 110 may send a sensorcapture confirmation signal to a computing device associated with theturntable 405, such as dynamic scene control bridge 710 and/or thecomputing device 745 of the turntable system 405. The sensor captureconfirmation signal may then be used as a signal that the turntable 405is allowed to begin (and/or should begin) rotation of the platform 420about the base 425 from the second rotation orientation to a nextrotation orientation. Otherwise, if sensor data capture is not completestep 1225 follows step 1230.

If, at step 1235, the internal computing device 110 of the vehicle 102determines that sufficient data has been captured by the vehicle 102'ssensors 180 to perform calibration—then no more rotations of theplatform 420 and the vehicle 102 about the base 425 are needed and step1240 follows step 1235, thus proceeding from sensor data capture tosensor calibration. Optionally, the vehicle computing system 110 maysend a sensor capture completion signal to a computing device associatedwith the turntable 405, such as dynamic scene control bridge 710 and/orthe computing device 745 of the turntable system 405. The sensor capturecompletion signal may then be used as a signal that the platform 420 ofthe turntable 405 is allowed to stop (and/or should stop) rotating aboutthe base 425 altogether to allow the vehicle 102 to exit the turntable405 and the calibration environment, or that the platform 425 of theturntable 405 is allowed to begin (and/or should begin) rotating aboutthe base 425 to an exit orientation that allow the vehicle 102 to exitthe turntable and the calibration environment (for example when thecalibration environment includes many targets around the turntable 405except for in an entrance/exit direction, as in FIG. 6 where an optimalentrance/exit direction is on the bottom-right due to lack of targetsand obstacles generally in that direction). Otherwise, if the internalcomputing device 110 of the vehicle 102 does not determine thatsufficient data has been captured by the vehicle 102's sensors 180 toperform calibration at step 1235, then step 1210 follows after step1235, to continue rotations of the platform 420 (and vehicle 102) aboutthe base 425 of the motorized turntable system 405. Depending on thesensors 180 on the vehicle 102 and the data captured by the sensors 180,the sensors 180 may require one or more full 360 degree rotations of thevehicle 102 on the platform 420, or may require less than one full 360degree rotation of the vehicle 102 on the platform 420. In oneembodiment, sufficient data for calibration of a sensor may mean datacorresponding to targets covering at least a subset of the completefield of view of a particular sensor (collectively over a number ofcaptures), with the subset reaching and/or exceeding a thresholdpercentage (e.g., 30%, 35%, 40%, 45%, 50%, 55%, 60%, 65%, 70%, 75%, 80%,85%, 90%, 95%, 99%, 100%).

Some sensors may require more data for calibration than others, andthus, one sensor may have captured sufficient data for calibration whileanother sensor might not. In such cases, step 1235 may refer to allsensors and thus go through the “NO” arrow if any of the sensors 180hasn't captured sufficient data. Alternately, a particular sensorcapturing sufficient data, or a majority of sensors capturing sufficientdata, may be the deciding factor toward “YES” or “NO.” In some cases,step 1235 may refer to each sensor separately, and once a particularsensor has captured sufficient data at step 1235, that sensor maycontinue on to step 1240 for calibration even if the vehicle 102 on theplatform 420 continues to rotate about the base 425 and the remainingsensors continue to capture data. Thus, step 1235 may enable staggeredcompletion of capture of sufficient data for different sensors atdifferent times.

In some cases, sensor data capture and sensor calibration occurs atleast partially in parallel; that is, a time period in which sensor datacapture occurs may at least partially overlap with a time period inwhich sensor calibration occurs. In such cases, the sensor may calibrateregion by region, for example by calibrating the sensor in one or moreregions in which the sensor detects (e.g., “sees”) targets for each datacapture until the entire point of view of the sensor, or some sufficientsubset is calibrated, with the subset reaching and/or exceeding athreshold percentage (e.g., 30%, 35%, 40%, 45%, 50%, 55%, 60%, 65%, 70%,75%, 80%, 85%, 90%, 95%, 99%, 100%). Calibration of each sensor may usea different threshold, or certain sensors may share a threshold. Whencalibration occurs in parallel with capture rather than calibration onthe whole sequentially following capture on the whole, step 1235 cansimply identify when calibration of one or more sensors has successfullycompleted, and treat that as a proxy for identifying whether sufficientdata is captured by those sensors to perform calibration.

In this case, however, step 1210 now rotates the vehicle 102 from thesecond position to a third position, and steps 1225 and 1230 refer tothe third position. The next time step 1210 is reached in this fashion,it now rotates the vehicle 102 from the third position to a fourthposition, and so on. In this way, step 1210 rotates the vehicle 102 onthe platform 420 about the base 425 from its current rotationalorientation/position to its next rotational orientation/position.

At step 1240, the internal computing device 110 of the vehicle 102proceeds on from sensor data capture to actual calibration of thesensors, for example as in steps 1025-1045 of FIG. 10 or steps 1125-1150of FIG. 11. To clarify, as discussed further above, capturing data viathe sensors 180 as in steps 1225-1235 and calibrating the sensors 180 asin step 1240 can be performed in parallel, between rotations of theplatform 420 and vehicle 102 about the base 425, or in any order thatcauses the process to be efficient. That is, calibration of datacaptured by a given one of the sensors 180 can begin immediately afterthe given sensor captures any new data, and can continue while thevehicle 102 is rotating about the base 425 on the platform 420 of theturntable 405 and while the sensors 180 capture further data. Becausecalibration and capture may be staggered and/or parallelized so thatsome further capture occurs after some calibration has started, dashedarrows extend from step 1240 to steps 1210 and 1225.

It should be understood that many of the steps of FIG. 12 (such as 1205,1215, 1220, 1230, and 1235) are optional.

FIG. 13 is a flow diagram illustrating operations for detection of, andcompensation for, a sloped turntable surface.

At optional step 1305, the turntable 405, vehicle 102, or surveyingsystem 610 identifies that the vehicle 102 is positioned on platform 420of the motorized turntable 405 as in step 1205 of FIG. 12, for examplebased on pressure sensors in the turntable, positioning receivers of thevehicle 102, and/or on the scene surveying system's visualidentification of the vehicle 102's position. Step 1305 (or 1205) mayoptionally include identifying that the vehicle 102 is specifically in adefined position on the platform 420 such as the defined positionidentified in FIG. 8B.

At step 1310, the one or more sensors 180 of the vehicle 102 are used tocheck whether the vehicle 102 is level or on a slope. The vehicle 102may use an IMU, one or more levels, one or more accelerometers, one ormore gyroscopes, one or more lidar sensor, one or more radar sensors,one or more sonar sensors, one or more sodar sensor, one or morecameras, any other sensor discussed herein, or some combination thereofto check whether the vehicle 102 is level or on a slope. For example,the vehicle 102 may use one or more gyroscopes, such as one or moregyroscopes found that are part of an IMU of the vehicle 102, to comparethe angle of the vehicle 102 while the vehicle 102 is on the platform420 to a reference angle of each gyroscope, the reference angle of thegyroscope corresponding to a level slope. The vehicle 102 mayalternately or additionally use one or more range sensors, such as oneor more lidar, radar, sonar, or sodar sensors, to identify the slope ofthe vehicle 102 based on on range from each range sensor to differentpoints along the platform 420 and/or to different points along a floorthat the turntable 420 rests on, where the different points should beequidistant if the vehicle 102 (and thus the turntable 405) is level, orwhere the different points should have a specific proportionalrelationship if the vehicle 102 (and thus the turntable 405) is level.The vehicle 102 may alternately or additionally use images from one ormore cameras, for example to identify a horizon in the one or moreimages, to identify a slope of the vehicle 102 (and thus the turntable405). With cameras, a slope of the floor may be detectable by checkingslopes of various edges of the calibration environment that should beparallel or perpendicular in X, Y, or Z dimensions (e.g., edge betweenfloor and wall, edge between one wall and another wall, edge betweenwall and ceiling) against each other to see if any unexpected anglesemerge, which may indicate a slope. Sensors 735 within the turntable 405itself, such as any of those described above with respect to the vehicle102 in step 1310, may alternately or additionally be used to detect theslope of the turntable 405 (and thus the vehicle) instead of oralongside the sensors 180 of the vehicle 102.

If, at step 1315, the internal computing device 110 of the vehicle 102identifies that the vehicle 102 (and therefore the turntable 405) islevel while vehicle is at this position, then step 1315 is followed bystep 1325; otherwise, step 1315 is followed by step 1320. If the vehicle102 (and/or turntable 405) uses a gyroscope measurement to determineslope, then the gyroscope angle measurement can be compared to agyroscope reference angle of the gyroscope, and if a difference betweenthe two is found to exceed an angle threshold (e.g., N degrees, where Nis 0, 0.5, 1, 1.5, 2, 2.5, 3, 3.5, 4, 4.5, 5, 5.5, 6, 6.5, 7, 7.5, 8,8.5, 9, 9.5, or 10) then the slope is not level. If the vehicle 102(and/or turntable 405) uses a range sensor to determine slope, then oneor more differences from expected range values to different points alongthe floor exceeding a range threshold (e.g, N millimeters orcentimeters, where N is 0, 0.5, 1, 1.5, 2, 2.5, 3, 3.5, 4, 4.5, 5, 5.5,6, 6.5, 7, 7.5, 8, 8.5, 9, 9.5, or 10) may indicate that the slope isnot level. If the vehicle 102 (and/or turntable 405) uses a camera todetermine slope, then differences from expected angles in resultingimages that are found to exceed an angle threshold (e.g., N degrees,where N is 0, 0.5, 1, 1.5, 2, 2.5, 3, 3.5, 4, 4.5, 5, 5.5, 6, 6.5, 7,7.5, 8, 8.5, 9, 9.5, or 10) may indicate that the slope is not level. Atstep 1320, the internal computing device 110 of the vehicle 102 (and/orthe computing device 745 of the motorized turntable system 405)identifies a specific slope of vehicle 102 (and therefore a slope of theturntable 405) at the vehicle and turntable's current rotation position.At step 1325, the vehicle captures sensor datasets using its sensors180, optionally at multiple orientations, for example as in steps 910,1010, 1110, 1115, and/or 1225. In some cases, the slope may have alreadybeen identified at step 1310, at which case step 1325 may not entailperforming anything. In other cases, the slope may have been identifiedin a quick, imprecise manner at step 1310, while at step 1325, a moreaccurate measurement is determined, using any of the sensors of thevehicle 102 and/or of the motorized turntable system 405 discussed withrespect to step 1310. In some cases, for example, the motorizedturntable 405 may rotate the platform 420 about the base 425 while thevehicle 102 is on the platform 420, and the vehicle 102 may performadditional measurements at different orientations along this rotationand use those additional measurements to more precisely determine slope.These additional slope detection measurements at different orientationsmay occur at step 1310 and/or at step 1325, and may occur while theplatform 420 is stationary relative to the base 425 (for example duringpauses in rotation), while the platform 420 is rotating relative to thebase 425, or some combination thereof.

At step 1330, the vehicle 102 performs sensor calibration of itssensors, for example as in steps 915-925, 1025-1040, 1125-1150, and/or1240. The sensor calibration performed at step 1330 may be performedpartially (e.g., more sensor calibration capture data is yet to becollected, and/or the motorized turntable still has more rotations toperform) or to completion.

Note that, as discussed with respect to FIG. 12, capture of data by thesensors 180 of the vehicle 102 may occur in parallel with calibration ofthe sensors 180 of the vehicle 102. This may cause calibration andcapture to be staggered and/or parallelized so that some further captureoccurs after some calibration has started, represented by the dashedarrow from step 1330 to step 1325.

At step 1335, the internal computing device 110 of the vehicle 102additionally compensates for the slope identified at step 1320 in itscalibration of sensors at step 1330. For example, if the turntable isnot level surface, but is instead tilted along a north-south axis by onedegree, then all of the sensor results will be skewed along thenorth-south axis. The internal computing device 110 of the vehicle 102can calculate new positions of everything in the calibration environmentby tilting everything by one degree along the north-south axis in theopposite direction as the slope identified in step 1320.

At optional step 1340, the rotation from the current position to thenext position may be triggered, much like in step 1210 of FIG. 12. Step1340 may be triggered when the calibration performed at step 1330 isonly partial rather than complete, or when the slope is still notdetermined to a satisfactory degree (i.e., other angles should bechecked). The rotation of step 1340 may be followed by step 1310,detecting slope again after the rotation, which may be helpful sincecertain slopes may be more detectable while the vehicle 102 is in oneorientation versus another orientation due to the fact that vehicles 102are typically longer along their forward-backward axis than they arewide along their left-right axis and/or simply due to limitations of thesensor(s) used to determine slope at certain orientations that might notbe limiting at others. Detection of the slope may be detected, verified,and/or adjusted by the vehicle 102 and/or by the turntable system 405while the platform 420 is at various orientations with respect to thebase 425. Thus, slope detection may occur in parallel with othercalibration procedures illustrated in and/or discussed with respect toFIG. 9, FIG. 10, FIG. 11, FIG. 12, and/or FIG. 14. By repeating thisslope detection and compensation at a multiple rotation locations, theinternal computing device 110 of the vehicle 102 may develop a morenuanced understanding of how level or sloped the vehicle 102, and thusthe turntable 405, is at each position.

FIG. 14A is a flow diagram illustrating operations for interactionsbetween the vehicle and a lighting system.

At step 1405, the vehicle 102 captures sensor datasets using its sensors180, for example as in steps 910, 1010, 1110, 1115, 1225, and/or 1325.At step 1410, the internal computing system 110 of the vehicle 102identifies whether a characteristic of one or more sensor targets—inthis case lighting conditions in at least one area of the calibrationenvironment that includes one or more sensor targets—are suboptimal, atleast for the purposes of calibration. In some cases, the computer 110of the vehicle 102 may identify that a representation of a sensor targetthat is identified within a sensor dataset (such as a photo or video)captured using the sensor (such as a camera) is suboptimal or notsuitable for calibration, for example because the sensor target is toodimly lit, too brightly lit, or lit from the wrong angle (e.g., causingglare, shadows, dimness, brightness, uneven lighting, or otherwiseaffecting the representation of the sensor target). Such lightingconditions may be suboptimal because they may cause a sensor to notproperly or clearly detect out one or more features of the sensortarget, such as a checkerboard pattern 210A or ArUco pattern 210B orcrosshair pattern 210C of a camera target 200, or a shape 215 of a radartarget 220, or a aperture 225/240 and/or marking 230 and/or target ID235 of a combined camera/depth sensor target 250.

If, at step 1410, the computer 110 of the vehicle 102 determines thatthe lighting conditions are suboptimal, then step 1410 is followed bystep 1415; otherwise, step 1410 is followed by step 1420, at which pointthe vehicle proceeds from capture to sensor calibration of its sensors,for example as in steps 915-925, 1025-1040, 1125-1150, 1240, and/or1330.

Note that, as discussed with respect to FIG. 12, capture of data by thesensors 180 of the vehicle 102 may occur in parallel with calibration ofthe sensors 180 of the vehicle 102. This may cause calibration andcapture to be staggered and/or parallelized so that some further captureoccurs after some calibration has started, represented by the dashedarrow from step 1420 to step 1405.

At step 1415, the internal computing system 110 of the vehicle 102 sendsan environment adjustment signal or message to an environment adjustmentsystem (in this case the lighting system 760) to activate one or moreactuators 762 and thereby adjust lighting conditions in the at least onearea (and affecting one or more sensor targets in the at least one area)of the calibration environment. The one or more actuators 762 maycontrol one or more motors associated with the lighting system 760, oneor more switches associated with the lighting system 760, and/or one ormore dimmers associated with the lighting system 760. Upon receiving theenvironment adjustment signal or message from the vehicle 102, thelighting system 760 can activate the one or more actuators 762, and canthereby effect a modification to the characteristic (i.e., the lightingcondition) of the one or more sensor targets, for example by brighteningone or more light sources 620, by dimming one or more light sources 620,by moving one or more light sources 620 translationally, by rotating oneor more light sources 620 (i.e., moving the one or more light sources620 rotationally), by activating (i.e., turning on) one or more lightsources 620, by deactivating (i.e., turning off) one or more lightsources 620, by changing a color emitted by (or filtered via colorfilters applied to) the one or more light sources 620, by otherwisemodifying the one or more light sources 620, or some combinationthereof. Note that an increase in brightness as discussed herein mayrefer to brightening one or more light sources 620, activating one ormore one or more light sources 620, and/or moving one or more lightsources 620. Note that a decrease in brightness as discussed herein mayrefer to dimming one or more light sources 620, deactivating one or moreone or more light sources 620, and/or moving one or more light sources620.

After step 1415, the process returns to 1405 to capture the sensor datawith newly-adjusted (i.e., optimized) lighting. The newly-adjustedlighting is then checked at step 1410 to see whether the adjustment fromstep 1415 corrected the lighting condition issue identified previouslyat step 1410 (leading to step 1420), or if further adjustments arerequired (leading to step 1415 once again).

FIG. 14B is a flow diagram illustrating operations for interactionsbetween the vehicle and a target control system.

At step 1425, the vehicle 102 captures sensor datasets using its sensors180, for example as in steps 910, 1010, 1110, 1115, 1225, 1325, and/or1405. At step 1430, the internal computing system 110 of the vehicle 102identifies whether a characteristic of one or more sensor targets—inthis case sensor target positioning of at least one target in thecalibration environment is suboptimal, at least for the purposes ofcalibration. In some cases, the computer 110 of the vehicle 102 mayidentify that a representation of a sensor target that is identifiedwithin a sensor dataset (such as a photo or video or radar image/videoor lidar image/video) captured using the sensor (such as a camera orradar or lidar sensor) is suboptimal or not suitable for calibration,for example because the sensor target is located in a position and/orfacing an orientation in which the sensor cannot properly or clearlydetect out one or more features of the sensor target, such as acheckerboard pattern 210A or ArUco pattern 210B or crosshair pattern210C of a camera target 200, or a shape 215 of a radar target 220, or aaperture 225/240 and/or marking 230 and/or target ID 235 of a combinedcamera/depth sensor target 250.

If, at step 1430, the computer 110 of the vehicle 102 determines thatthe sensor target positioning is sub-optimal, then step 1430 is followedby step 1435; otherwise, step 1430 is followed by step 1440, at whichpoint the vehicle proceeds from capture to sensor calibration of itssensors, for example as in steps 915-925, 1025-1040, 1125-1150, 1240,1330, and/or 1420.

Note that, as discussed with respect to FIG. 12, capture of data by thesensors 180 of the vehicle 102 may occur in parallel with calibration ofthe sensors 180 of the vehicle 102. This may cause calibration andcapture to be staggered and/or parallelized so that some further captureoccurs after some calibration has started, represented by the dashedarrow from step 1440 to step 1425.

At step 1435, the internal computing system 110 of the vehicle 102 sendsan environment adjustment signal or message to an environment adjustmentsystem (in this case the target control system 770) to activate one ormore actuators 774 and thereby move the at least one sensor target to amore optimal position in the calibration environment. The one or moreactuators 774 may control one or more motors associated with the targetcontrol system 770 and/or one or more switches associated with thetarget control system 770. Upon receiving the environment adjustmentsignal or message from the vehicle 102, the target control system 770can activate the one or more actuators 774, and can thereby effect amodification to the characteristic (i.e., the positioning) of the one ormore sensor targets, for example by activating one or more motors thattranslationally move one or more targets and/or by activating one ormore motors that rotate one or more targets (each about an axis).

After step 1435, the process returns to 1425 to capture the sensor datawith newly-moved (i.e., optimized) sensor target positioning. Thenewly-moved target positioning is then checked at step 1430 to seewhether the adjustment from step 1435 corrected the target positioningissue identified previously at step 1430 (leading to step 1440), or iffurther adjustments are required (leading to step 1435 once again).

In some cases, the adjustment(s) to lighting of FIG. 14A and theadjustment(s) to target positioning of FIG. 14B may both occur followingcapture of the same sensor dataset with the same sensor, In such cases,the checks of steps 1410 and 1430 may be performed repeatedly, onceafter each adjustment in either target positioning or lighting, sincemovement of a sensor target may correct or ameliorate issues withlighting, and on the other hand, adjustment of lighting may also corrector ameliorate issues with target positioning. In such cases, the sendingof messages, and the resulting adjustments, of steps 1415 and step 1435,can either be performed sequentially (and then tested at steps 1410and/or 1430), or can be performed in parallel (and then tested at steps1410 and/or 1430).

While various flow diagrams provided and described above, such as thosein FIGS. 9, 10, 11, 12, 13, 14A, and 14B, may show a particular order ofoperations performed by some embodiments of the subject technology, itshould be understood that such order is exemplary. Alternativeembodiments may perform the operations in a different order, combinecertain operations, overlap certain operations, or some combinationthereof. It should be understood that unless disclosed otherwise, anyprocess illustrated in any flow diagram herein or otherwise illustratedor described herein may be performed by a machine, mechanism, and/orcomputing system #00 discussed herein, and may be performedautomatically (e.g., in response to one or more triggers/conditionsdescribed herein), autonomously, semi-autonomously (e.g., based onreceived instructions), or a combination thereof. Furthermore, anyaction described herein as occurring in response to one or moreparticular triggers/conditions should be understood to optionally occurautomatically response to the one or more particulartriggers/conditions.

As described herein, one aspect of the present technology is thegathering and use of data available from various sources to improvequality and experience. The present disclosure contemplates that in someinstances, this gathered data may include personal information. Thepresent disclosure contemplates that the entities involved with suchpersonal information respect and value privacy policies and practices,for example by encrypting such information.

FIG. 15 shows an example of computing system 1500, which can be forexample any computing device making up internal computing system 110,remote computing system 150, (potential) passenger device executingrideshare app 170, or any component thereof in which the components ofthe system are in communication with each other using connection 1505.Connection 1505 can be a physical connection via a bus, or a directconnection into processor 1510, such as in a chipset architecture.Connection 1505 can also be a virtual connection, networked connection,or logical connection.

In some embodiments, computing system 1500 is a distributed system inwhich the functions described in this disclosure can be distributedwithin a datacenter, multiple data centers, a peer network, etc. In someembodiments, one or more of the described system components representsmany such components each performing some or all of the function forwhich the component is described. In some embodiments, the componentscan be physical or virtual devices.

Example system 1500 includes at least one processing unit (CPU orprocessor) 1510 and connection 1505 that couples various systemcomponents including system memory 1515, such as read-only memory (ROM)1520 and random access memory (RAM) 1525 to processor 1510. Computingsystem 1500 can include a cache of high-speed memory 1512 connecteddirectly with, in close proximity to, or integrated as part of processor1510.

Processor 1510 can include any general purpose processor and a hardwareservice or software service, such as services 1532, 1534, and 1536stored in storage device 1530, configured to control processor 1510 aswell as a special-purpose processor where software instructions areincorporated into the actual processor design. Processor 1510 mayessentially be a completely self-contained computing system, containingmultiple cores or processors, a bus, memory controller, cache, etc. Amulti-core processor may be symmetric or asymmetric.

To enable user interaction, computing system 1500 includes an inputdevice 1545, which can represent any number of input mechanisms, such asa microphone for speech, a touch-sensitive screen for gesture orgraphical input, keyboard, mouse, motion input, speech, etc. Computingsystem 1500 can also include output device 1535, which can be one ormore of a number of output mechanisms known to those of skill in theart. In some instances, multimodal systems can enable a user to providemultiple types of input/output to communicate with computing system1500. Computing system 1500 can include communications interface 1540,which can generally govern and manage the user input and system output.The communication interface may perform or facilitate receipt and/ortransmission wired or wireless communications via wired and/or wirelesstransceivers, including those making use of an audio jack/plug, amicrophone jack/plug, a universal serial bus (USB) port/plug, an Apple®Lightning® port/plug, an Ethernet port/plug, a fiber optic port/plug, aproprietary wired port/plug, a BLUETOOTH® wireless signal transfer, aBLUETOOTH® low energy (BLE) wireless signal transfer, an IBEACON®wireless signal transfer, a radio-frequency identification (RFID)wireless signal transfer, near-field communications (NFC) wirelesssignal transfer, dedicated short range communication (DSRC) wirelesssignal transfer, 802.11 Wi-Fi wireless signal transfer, wireless localarea network (WLAN) signal transfer, Visible Light Communication (VLC),Worldwide Interoperability for Microwave Access (WiMAX), Infrared (IR)communication wireless signal transfer, Public Switched TelephoneNetwork (PSTN) signal transfer, Integrated Services Digital Network(ISDN) signal transfer, 3G/4G/5G/LTE cellular data network wirelesssignal transfer, ad-hoc network signal transfer, radio wave signaltransfer, microwave signal transfer, infrared signal transfer, visiblelight signal transfer, ultraviolet light signal transfer, wirelesssignal transfer along the electromagnetic spectrum, or some combinationthereof. The communications interface 1540 may also include one or moreGlobal Navigation Satellite System (GNSS) receivers or transceivers thatare used to determine a location of the computing system 1500 based onreceipt of one or more signals from one or more satellites associatedwith one or more GNSS systems. GNSS systems include, but are not limitedto, the US-based Global Positioning System (GPS), the Russia-basedGlobal Navigation Satellite System (GLONASS), the China-based BeiDouNavigation Satellite System (BDS), and the Europe-based Galileo GNSS.There is no restriction on operating on any particular hardwarearrangement, and therefore the basic features here may easily besubstituted for improved hardware or firmware arrangements as they aredeveloped.

Storage device 1530 can be a non-volatile and/or non-transitory and/orcomputer-readable memory device and can be a hard disk or other types ofcomputer readable media which can store data that are accessible by acomputer, such as magnetic cassettes, flash memory cards, solid statememory devices, digital versatile disks, cartridges, a floppy disk, aflexible disk, a hard disk, magnetic tape, a magnetic strip/stripe, anyother magnetic storage medium, flash memory, memristor memory, any othersolid-state memory, a compact disc read only memory (CD-ROM) opticaldisc, a rewritable compact disc (CD) optical disc, digital video disk(DVD) optical disc, a blu-ray disc (BDD) optical disc, a holographicoptical disk, another optical medium, a secure digital (SD) card, amicro secure digital (microSD) card, a Memory Stick® card, a smartcardchip, a EMV chip, a subscriber identity module (SIM) card, amini/micro/nano/pico SIM card, another integrated circuit (IC)chip/card, random access memory (RAM), static RAM (SRAM), dynamic RAM(DRAM), read-only memory (ROM), programmable read-only memory (PROM),erasable programmable read-only memory (EPROM), electrically erasableprogrammable read-only memory (EEPROM), flash EPROM (FLASHEPROM), cachememory (L1/L2/L3/L4/L5/L#), resistive random-access memory (RRAM/ReRAM),phase change memory (PCM), spin transfer torque RAM (STT-RAM), anothermemory chip or cartridge, and/or a combination thereof.

The storage device 1530 can include software services, servers,services, etc., that when the code that defines such software isexecuted by the processor 1510, it causes the system to perform afunction. In some embodiments, a hardware service that performs aparticular function can include the software component stored in acomputer-readable medium in connection with the necessary hardwarecomponents, such as processor 1510, connection 1505, output device 1535,etc., to carry out the function.

For clarity of explanation, in some instances, the present technologymay be presented as including individual functional blocks includingfunctional blocks comprising devices, device components, steps orroutines in a method embodied in software, or combinations of hardwareand software.

Any of the steps, operations, functions, or processes described hereinmay be performed or implemented by a combination of hardware andsoftware services or services, alone or in combination with otherdevices. In some embodiments, a service can be software that resides inmemory of a client device and/or one or more servers of a contentmanagement system and perform one or more functions when a processorexecutes the software associated with the service. In some embodiments,a service is a program or a collection of programs that carry out aspecific function. In some embodiments, a service can be considered aserver. The memory can be a non-transitory computer-readable medium.

In some embodiments, the computer-readable storage devices, mediums, andmemories can include a cable or wireless signal containing a bit streamand the like. However, when mentioned, non-transitory computer-readablestorage media expressly exclude media such as energy, carrier signals,electromagnetic waves, and signals per se.

Methods according to the above-described examples can be implementedusing computer-executable instructions that are stored or otherwiseavailable from computer-readable media. Such instructions can comprise,for example, instructions and data which cause or otherwise configure ageneral purpose computer, special purpose computer, or special purposeprocessing device to perform a certain function or group of functions.Portions of computer resources used can be accessible over a network.The executable computer instructions may be, for example, binaries,intermediate format instructions such as assembly language, firmware, orsource code. Examples of computer-readable media that may be used tostore instructions, information used, and/or information created duringmethods according to described examples include magnetic or opticaldisks, solid-state memory devices, flash memory, USB devices providedwith non-volatile memory, networked storage devices, and so on.

Devices implementing methods according to these disclosures can comprisehardware, firmware and/or software, and can take any of a variety ofform factors. Typical examples of such form factors include servers,laptops, smartphones, small form factor personal computers, personaldigital assistants, and so on. The functionality described herein alsocan be embodied in peripherals or add-in cards. Such functionality canalso be implemented on a circuit board among different chips ordifferent processes executing in a single device, by way of furtherexample.

The instructions, media for conveying such instructions, computingresources for executing them, and other structures for supporting suchcomputing resources are means for providing the functions described inthese disclosures.

Although a variety of examples and other information was used to explainaspects within the scope of the appended claims, no limitation of theclaims should be implied based on particular features or arrangements insuch examples, as one of ordinary skill would be able to use theseexamples to derive a wide variety of implementations. Further andalthough some subject matter may have been described in languagespecific to examples of structural features and/or method steps, it isto be understood that the subject matter defined in the appended claimsis not necessarily limited to these described features or acts. Forexample, such functionality can be distributed differently or performedin components other than those identified herein. Rather, the describedfeatures and steps are disclosed as examples of components of systemsand methods within the scope of the appended claims.

What is claimed is:
 1. A system for vehicle environment control, thesystem comprising: an actuator in a calibration environment; acommunication interface; a memory storing instructions; and a processorthat executes the instructions, wherein execution of the instructions bythe processor causes the processor to: receive an environment adjustmentsignal from a vehicle computing system of a vehicle in the calibrationenvironment via the communication interface; activate the actuatorautomatically in response to receiving the environment adjustment signalfrom the vehicle computing system, wherein activating the actuatoreffects an adjustment to a characteristic of a sensor target in thecalibration environment; and send a confirmation signal to the vehiclecomputing system via the communication interface, the confirmationsignal indicative of the adjustment to the characteristic of the sensortarget.
 2. The system of claim 1, wherein the environment adjustmentsignal identifies the characteristic.
 3. The system of claim 1, whereinthe environment adjustment signal identifies the adjustment.
 4. Thesystem of claim 1, wherein the environment adjustment signal indicatesthat a representation of the sensor target in a sensor capture datasetis suboptimal for calibration, wherein the sensor capture dataset iscaptured by a sensor of the vehicle.
 5. The system of claim 1, whereinthe characteristic of the sensor target includes a lighting condition ofthe sensor target, wherein activation of the actuator includes adjustinga light source, and wherein the adjustment to the characteristicincludes a modification to the lighting condition of the sensor target.6. The system of claim 5, wherein the characteristic of the sensortarget includes a brightness of the sensor target, wherein adjusting thelight source includes modifying a brightness of the light source, andwherein the modification to the lighting condition of the sensor targetincludes a modification to the brightness of the sensor target.
 7. Thesystem of claim 5, wherein the characteristic of the sensor targetincludes a direction of light affecting the sensor target, whereinadjusting the light source includes moving the light source, and whereinthe modification to the lighting condition of the sensor target includesa modification to the direction of light affecting the sensor target. 8.The system of claim 5, further comprising: the light source.
 9. Thesystem of claim 1, wherein the characteristic of the sensor targetincludes a positioning of the sensor target, wherein activation of theactuator actuates a motor, and wherein the adjustment to thecharacteristic includes moving the sensor target from a first sensortarget position to a second sensor target position via the motor. 10.The system of claim 9, wherein moving the sensor target from a firstsensor target position to a second sensor target position via the motorincludes rotating the sensor target about an axis.
 11. The system ofclaim 9, wherein moving the sensor target from a first sensor targetposition to a second sensor target position via the motor includeseffecting a translational movement of the sensor target along a definedmovement path.
 12. The system of claim 9, further comprising: the motor.13. The system of claim 1, further comprising: the sensor target. 14.The system of claim 1, wherein execution of the instructions by theprocessor causes the processor to: actuate a motor associated with amotorized turntable to rotate a platform of the motorized turntableabout a base of the motorized turntable, wherein the vehicle isconfigured to rest atop the platform of the motorized turntable whilethe platform of the motorized turntable rotates about the base of themotorized turntable, wherein actuating the motor associated with themotorized turntable rotates the vehicle from a first position to asecond position.
 15. The system of claim 14, wherein receiving theenvironment adjustment signal from the vehicle computing system via thecommunication interface includes receiving the environment adjustmentsignal from the vehicle computing system via the communication interfacewhile the vehicle is in the first position.
 16. The system of claim 14,wherein receiving the environment adjustment signal from the vehiclecomputing system via the communication interface includes receiving theenvironment adjustment signal from the vehicle computing system via thecommunication interface while the vehicle is in the second position. 17.The system of claim 14, further comprising: the motorized turntable; andthe motor associated with the motorized turntable.
 18. The system ofclaim 1, wherein the communication interface includes a wirelesscommunication interface, wherein receiving the environment adjustmentsignal from the vehicle computing system via the communication interfaceincludes receiving the environment adjustment signal from the vehiclecomputing system via the wireless communication interface, whereinsending the confirmation signal to the vehicle computing system via thecommunication interface includes sending the confirmation signal to thevehicle computing system via the wireless communication interface.
 19. Amethod of vehicle environment control, the method comprising: receivingan environment adjustment signal from a vehicle computing system of avehicle in a calibration environment via a communication interface;activating an actuator automatically in response to receiving theenvironment adjustment signal from the vehicle computing system, whereinactivating the actuator effects an adjustment to a characteristic of asensor target in the calibration environment; and sending a confirmationsignal to the vehicle computing system via the communication interface,the confirmation signal indicative of the adjustment to thecharacteristic of the sensor target.
 20. A non-transitory computerreadable storage medium having embodied thereon a program, wherein theprogram is executable by a processor to perform a method of vehicleenvironment control, the method comprising: receiving an environmentadjustment signal from a vehicle computing system of a vehicle in acalibration environment via a communication interface; activating anactuator automatically in response to receiving the environmentadjustment signal from the vehicle computing system, wherein activatingthe actuator effects an adjustment to a characteristic of a sensortarget in the calibration environment; and sending a confirmation signalto the vehicle computing system via the communication interface, theconfirmation signal indicative of the adjustment to the characteristicof the sensor target.